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PREFACE 


Tl  3  RAND /TAG  information  and  analysis  system  provides  for  the 
collection,  processing  and  analysis  of  operations,  maintenance  and 
supply  data,  using  an  IBM  1401  computer  to  assist  in  data  purifica¬ 
tion  and  in  the  management  and  evaluation  of  aircraft  operations  and 
support  at  base  level.  The  system  is  unique  in  that  the  data  collected 
are  identified  with  the  specific  sortie  (and  in  some  instances  to  the 
specific  leg  within  the  sortie).  This  allows  the  user  to  perform  many 
kinds  of  analyses  not  ordinarily  p  ssible,  relating  mission  use  to 
reliability,  manpower  and  spares  usage. 

The  system  grew  out  of  a  number  of  special  field  tests  (e.g., 

Rapid  Roger,  Skoshi  Tiger,  Tack  D^wn)  exploring  the  feasibility  of 
using  a  small  business  computer  to  assist  with  materiel  and  operations 
problems  at  base  level.  In  the  tests  prior  to  Combat  Dragon,  it  was 
necessary  to  "create"  the  maintenance  analysts  through  an  extensive 
educational  process.  Combat  Dragon  was  unique  in  that  Air  Force  per¬ 
sonnel  carried  out  the  entire  project  from  data  collection  through 
final  writeup  with  no  assistance  from  RAND,  other  than  the  initial 
training  in  the  use  of  the  system. 

It  is  now  possible  to  organize  the  loose  collection  of  notes,  pro¬ 
cedures  and  programs  into  a  formal  system  description.  Accordingly,  this 
RAND  effort  comprises  four  Memorandums  containing  essentially  the  package 
of  materials  used  in  training  the  Combat  Dragon  team.  The  information 
is  organized  as  follows:  Volume  I  (RM-5666-FR)  is  for  data  collectors 
and  editors  responsible  for  providing  the  data  bank  to  be  used  in  sub¬ 
sequent  analyses;  Volumes  II  (RM-5667-l'R)  and  III  (RM-5668-PR)  are  for 
analysts  (especially  people  who  will  be  doing  maintenance  analysis)  to 
familiarize  them  with  the  available  programs  and  analysis  products,  and 
to  encourage  them  to  ask  questions  arid  explore  the  data  in  an  imaginative 
way;  and  Volume  IV  (RM-5669-PR)  is  for  the  "data  services  branch"  of  the 
evaluation  or  analysis  team,  to  identify  procedures  and  to  unpart  an 
understanding  of  what  the  analyst  is  attempting  to  do. 

Surprisingly,  even  though  the  system  is  entirely  computerized, 
readers  need  noc  have  a  knowledge  of  computer  hardware  and  software  to 
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follow  the  text.  A  knowledge  of  the  details  of  aircraft  weapon 
systems  would  be  useful,  for  although  we  describe  such  operations, 
the  descriptions  are  somewhat  cursory.  In  particular,  a  familiarity 
with  aircraft  maintenance  procedures  would  be  useful. 

The  concepts,  techniques  and  programs  of  the  RAND /T AC  information 
and  analysis  system  should  be  adaptable  to  future  Air  Fo.ce  base-level 
management  information  systems,  whether  manual  or  highly  mechanized. 
Provided  that  the  appropriate  computer  is  available,  the  RAND/TAC  sys¬ 
tem  can  easily  be  introduced  at  a  base  and  used  without  modification 
for  field  tests  or  other  purposes.  Recent  changes  in  a  number  of 
standard  Air  Force  forms  and  in  data  li'-'ages,  however,  may  make  them 
preferable  to  the  RAND/TAC  forms  for  a  particular  b«6e  exercise. 

With  modest  changes  to  current  Air  Force  data  collection  procedures 
and  reprogramming  of  the  analysis  packages,  the  system  would  provide  a 
valuable  supplement  to  current  base  analysis  reports--a  supplement  more 
attuned  to  questions  that  are  and  should  be  asked  by  base  maintenance 
management..  The  system  will  also  provide  a  detailed  guide  and  check 
list  for  the  design-development  of  new  base- level  information  aystems 
and  should  provide  direct  input  to  analysis  portions  thereof. 


SUMMARY 


The  RAND/TAC  information  and  analysis  system  provides  for  the  col¬ 
lection,  processing  and  an  ysis  of  operations,  maintenance  and  supply 
data,  using  a  small  business  computer  to  assist  in  data  purification 
and  in  the  management  and  evaluation  of  aircraft  opera  ms  and  support 
at  base  level.  It  is  unique  in  that  operational  and  logistics  variables 
are  interrelated  through  several  features  of  the  data  and  analysis 
systems  to  permit  identification  of  operational  events  connected  with 
a  particular  sortie  and  relate  these  to  explicit  maintenance  or  supply 
actions  preceding  or  following  the  sortie,  management  actions,  and  key 
environmental  conditions. 

The  system  consists  of  a  series  of  forms  for  collecting  perational 
data,  maintenance  actions,  maintenance  manpower  availability,  aerospace 
ground  equipment  utilization,  supply  demand,  cannibalization  and  issue 
data,  a  series  of  computer  programs  and  manual  procedures  for  editing, 
reformatting  and  processing  to  provide  basic  displays,  and  other  pro¬ 
grams  to  provide  basic  analysis  pac. ages,  The  system  is  designed  to 
minimize  duplicative  recording  of  •  ita  elements,  and  has  flexible  com¬ 
puter  programs  to  permit  a  wide  variety  of  analyses. 

The  four  volumes  constituting  this  effort  present  a  complete  sys¬ 
tem  description,  together  with  instructions  on  how  to  perform  analyses 
using  the  system  programs.  Volume  I  (RM-5666-PR)  contains  the  descrip¬ 
tions  of  and  procedures  for  collecting  and  editing  the  data--the  forms, 
procedures  and  program  operating  instructions.  Volumes  II  (RM-5667-PR) 
and  III  (RM-5b68-PR)  are  concerned  with  the  analysis  program*  and  pro¬ 
cedures,  and  with  analysis  design  id  methods.  The  firs  emphasizes 
how  the  programs  work,  the  second  how  questions  can  be  answered.  In  a 
sense  both  volumes  are  written  for  a  career  thaf  currently  does  not 
exist  in  the  Air  Force:  the  main'enar  analog  of  the  operation,  unalys 
A  person  interested  in  this  field  should  be  versed  not  only  in  maintenan 
but  also  in  data  processing,  computers,  statistical  methods  and  experi¬ 
mental  design. 

Each  time  RAND  participated  in  a  special  tield  test,  such  as  Com¬ 
bat  Dragon,  Skoshi  Tiger,  and  Tack  Down,  ii  was  necessary  to  "create” 
the  maintenance  analyats  by  an  extensive  educations!  process.  Volume 
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II  attempts  to  encapsulate  the  first  part  of  that  educational  process. 
It  introduces  the  prospective  analysts  to  the  data  bank,  the  programs 
and  the  procedure  needed  to  process  the  operations,  maintenance  and 
supply  analysis  data. 

Volume  III  is  based  on  the  second  stage  of  the  learning  nrocess. 
It  assumes  that  the  user  has  now  mastered  the  elements  of  the  program 
and  can  focus  his  attention  on  answering  questions.  Thus  it  addresses 
analysis  fundamentals:  dependent  and  independent  variables,  data 
fields,  sorting,  data  selection  and  tagging.  Then  it  discusses  a 
variety  of  areas  of  interest  to  maintenance  management  and  shows  how 
each  can  be  explored  with  the  system.  Finally,  some  of  Liie  twKgrouuu 
and  philosophy  of  experimental  design  is  discussed. 

Volume  IV  (RM-5669-PR)  describes  the  computer  programs  used  with 
the  system.  To  encourage  a  rapprochement  between  the  analysts  and  the 
programmers,  we  have  attempted  to  include  sufficient  information  for 
the  programmer  to  understand  the  general  outlines  of  whet  the  analyst 
is  attempting,  as  reflected  by  the  functions  of  the  computer  programs. 
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GLOSSARY 


Abort  rate 

Action  taken 

AGE 

AGE  utilization 
Aircraft  display 

Base-line 

Break- rates 


Chi-square 

Code  799 
Code  800 
Code  T 
Code  U 

Combat  Dragon 
Cost-effectiveness 

Daily  package 

Delay  time 
Demand  rates 


The  rate  of  aborted  sorties  made  by  a  unit  in 
a  given  period.  The  most  frequently  used 
equation  is  the  following:  abort  rate  «* 
sorties  aborted/sorties  attempted. 

The  type  of  maintenance  per?  >rmed:  repaired, 
removed  and  replaced,  calib;  ited,  etc. 

Aerospace  ground  equipment. 

A  display  showing  both  acti  e  and  standing 
flignt-1  ne  utilization  of  AGE. 

Also  cal  ed  flight-line  display  and  flight  - 
line  que’c  sort.  Shows  pictorially  the 
maintenance  and  status  history  of  each  air¬ 
craft  for  a  24-hour  period. 

Such  data  are  counts  of  the  times  an  event  was 
attempted,  e.g.,  sorties  flown  in  a  certain 
category. 

Along  with  write-ups,  break-rates  are  the  major 
independent  variables  in  determining  aircraft 
recovery  and  turnaround,  and  are  therefore  the 
major  determiners  of  sortie  generation  capability. 
Break-rates  are  determined  for  both  aircraft 
and  aircraft  systems.  The  equation  is  as 
follows:  break-rate  =  system  fix  count /sorties 
f lown. 

An  analysis  program  that  makes  statistical 
comparisons  of  frequency  counts  to  deter  ine 
whether  nonrandom  behavior  xists. 

No  defect  discovered. 

Removed  or  replaced  to  facilitate  maintenance. 

Removed  for  cannibalization. 

Replaced  after  canniba lizat ’  >n. 

An  exercise  with  A-37A  aircraft  in  Vietnam. 

An  exercise  with  F-4C  aircr  ;ft  at  MacDil  Air 
Force  Base. 

Processing  of  operations,  naintonance,  s  pply 
and  scheduling  data  by  means  of  edit,  er.'or 
listing,  and  aircraft  and  work  center  ili  splays. 

The  accumulation  o;  work  interruptions  t  t  each 
system  and  each  wo-  k  center,  by  aircraft  . 

Most  frequently,  supply  demand  rates.  Fast 
frequently,  demand  rate  =  demand  (reque „) / 
sorties  flown 
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Deviat ion-Dcgradat ion  (DEVDEG) 
program 

Dichotomous  data 


Easy  data 
Edit  mask  card 
Ed’t  program 

Eight/'two  pocket 
Eighty-eighty  (80-80)  listing 

Error  listing 


A  computer  program.  Lists  and  counts  missions' 
deviation-degradation  data  from  the  308  forms. 

Data  that  has  only  tv.o  c.ttegor  ies :  for  example, 
yes-no,  0-1,  hit-miss,  abort^d-nonaborted, 
ma  1  f  unct  ioned-nonma  1  f  ur.c  t  ioned . 

Data  co lie  ted  by  one  person  at  a  single  col¬ 
lection  point,  as  opposed  to  tough,  data. 

Locates  the  decimal  phase  that  is  edited  into 
the  data  field  just  before  printing. 

Searches  for  and  identifies  errors,  reformats 
data,  relieves  data  recorder  of  all  possible 
unnecessary  burden. 

Card  drop  pocket  on  the  1401. 

Records  are  printed  as  they  exist  on  the  card 
(or  tape)  without  separation  of  the  fields. 

An  image  of  each  card  containing  a  computer- 
detected  error  plus  a  description  and  location 
of  each  error. 


ETiC 

ETR  delay  code 
Field  location  card 


Estimated  time  to  in- commiss ion  status. 

Equipment  temporarily  removed. 

Locates  low-  and  high-order  positions  of  the 
fields  on  the  record. 


Flight-line  display 


See  aircraft  display. 


Frequency  Counter  (FREQ) 
program 


Searches  any  field  of  unknown  and  unsorted  data, 
builds  a  table,  and  counts  the  frequency  of 
entries  in  the  field. 


Gang  punch  To  punch  ideal1  al  or  c  astant  information  into 

all  of  a  file  of  cards. 

Gross  fix  time  Period  from  touchdown  to  end  of  maintenance. 

Includes  unscheduled  maintenance  only. 

Gross  turnaround  Period  from  touchdown  to  end  of  maintenance, 

includes  both  scheduled  and  unscheduled 
maintenance. 


Hard  d  ita 
Harmon  it  mean 


Hourly  Frequency  Accumulate 


How  malfunction  ode 
(how  mal) 


Keypunched  data,  as  opposed  to  soft  data. 

The  method  of  computing  helps  minimize  the  effect 
of  unequal  sample  size  by  using  the  reciprocal 
of  n. 

A  program  that  computes  the  1 requency  counts  for 
resource  utilization  for  each  of  the  24  hours 
during  the  day. 

Describes  the  nature  of  Hie  malfunction:  burned, 
distorted,  tracked,  overheated,  etc. 


I 


Independent  variables 


K-97  report 


Lag  tine 
Man-hours 

Manpower  Utilization  Sequence 


Mission  essential  items 


Net  fix  time 


Net  turnaround 


NORS-G 
N/P  pocket 


Observed  frequency 
Off-equipment  file 
On-equipment  bench  repair 


May  affect  system  behavior,  as  opposed  to 
dependent  variables,  which  arc  the  things 
being  affected. 

Ki  1  led  by  air , 

Deck  of  maintenance  data  forwarded  to  the 
Logistics  Command. 

Period  from  touchdown  until  maintenance  begins. 
Hours  of  direct  labor. 

Relates  personnel  utilization  (direct  on- 
equipment  labor)  to  sorties  flown. 

Manpower  utilization  Searches  each  minute 

of  the  hour  to  find  the 
number  of  men  working 
each  hour. 


Touchdown  counts 


Manpower  available  Produces  a  summary  card 

showing  the  number  of  men 
available  for  each  hour  of 
each  work  center  day. 

Touchdown  counts  Produces  a  card  for  each 

day  showing  the  number  of 
touchdowns  for  each  hour 
of  that  day. 

Essential  items  for  accomplishing  the  objective 
of  the  sortie. 

Maintenance  data  collection  portion  of  ATM  66-1. 
Maintenance  nondeliveries. 

Active  fix  time  when  work  is  being  accomplished. 
Lags  and  delays  are  not  included,  refers  only  to 
unschvdu  Ic-d  mai  ntenance . 

Same  as  net  fix  time  except  it  includes  the 
scheduled  maintenance  as  well. 

Not  operationally  ready,  maintenance. 

Not  operationally  ready,  supply. 

Not  operationally  ready  for  supply,  grounded. 

Card  drop  pocket  on  1401. 

Not  reparable  this  station. 

Count  of  successes,  failures,  aborts,  etc. 

MDC  records  of  bench  repair  ictions. 

Repair  done  without  item  going  through  conven¬ 
tional  materiel  control  channels. 
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OR  rate 

Oxnard  format 

Oxnard  project 

Pearson  product  moment 
correlations 

PS  code 

Quantitative  data 

Rapid  Roger 
Recombine  program 


Recovery  Sequence 


Recovery  Summary  (RECSITI) 
program 

SAC  Full  Force 

Sequential  Frequency 
Dist  ribution 

7- cards 

8- cards 


Operationally  ready  rate:  OR  rate  =  hours 
ready/hours  possessed. 

The  output  format  of  the  Edit  program  always 
includes  clock  hours.  Oxnard  refers  to  the 
project  for  which  the  format  was  designed. 

An  exercise  with  F-101  aircraft  at  Oxnard 
Air  Force  Base. 

Statistical  measure  showing  the  amount  of 
relationship  between  two  measures. 

Primary-secondary  code.  A  column  (in  record 
format  Ms)  for  use  as  a  squadron  (or  other) 
designator. 

Manhours,  elapsed  time.  Contrasts  with 
frequency  count  data. 

An  exercise  with  F-4C  aircraft  in  Thailand. 

A  special  purpose  program  used  in  the  Recovery 
Sequence.  Eliminates  duplications,  adds  a 
dummy  sortie  card  to  the  end  of  each  tail  num¬ 
ber  subset  and  merges  the  sortie  deck  output 
by  Single,  First  and  Last  with  the  nonsortie 
data  output  by  Compute  Elapsed  Time. 

Preprocessed  edit  output  data.  This  involves 
computing  elapsed  times,  converting  to  Julian 
Calendar,  and  coding  the  sortie  data  for  first, 
last  and  single  sorties  of  the  day.  Requires 
Col.  80=0,  =2,  =3  records.  Involves  four 
programs:  Compute  Elapsed  Time:  Jingle,  First, 
and  Last;  Recombine;  and  Clint. 

An  analysis  program.  Provides  a  complete, 
readily  comprehensible  summary  of  aircraft 
recovery  and  turnaround  in  a  one-page  general 
summary  with  back-up  pages  containin',  detail. 

An  exercise  with  B - 5 2 ,  B-47,  KC-13'  and  KC-97 
aircraft . 

Summarizes  and  displays  events  across  24-hour 
period . 

Produced  by  Clint  program.  Aircraft  recovery 
records  tlnit  include  only  the  unscheduled 
maintenance  action  and  the  postflights.  Air¬ 
craft  turnaround  records  that  include  both 
scheduled  and  unscheduled  maintenance  data. 

Produced  by  Clint  program.  System  records 
that  include  both  scheduled  and  unscheduled 
maintenance  actions. 


9-cards 


Produced  by  Cl<nt  program.  Work  center 
records  that  include  the  scheduled  and 
unscheduled  maintenance  actions  of  all  work 
centers . 


Sick  bird  analysis 

Single,  First  and  La^  (S/F'L) 
program 

Skoshi  Tiger 
Soft  data 
Sortie  length 
Sparrowhawk 
Sp lattergrams 


Determines  whether  individual  tail  numbers 
show  atypical  write-up  rates  based  on  the 
sorties  flown,  by  obtaining  the  sortie  and 
write-up  counts  for  each  aircraft  card,  using 
Chi-square  testing  for  nonhomogeniety, 

A  special  program  used  only  in  the  Recovery 
Sequence.  Makes  a  single  sortie  card  from 
the  pairs  of  sortie  cards  resulting  when  a 
flight  crosses  midnight.  The  program  also 
determines  and  tags  by  tail  number  the  sequence 
of  sorties  flown  each  day. 

An  exercise  with  F-4C,  F-5A,  and  F-100  aircraft 
in  Southeast  Asia. 

Data  not  keypunched,  as  opposed  to  hard  data. 
Generally  verbal  information. 

Measured  from  takeoff  to  chock  Lime  (engine 
shut-down) . 

An  exercise  with  F-4C.  F-5A,  and  A-4  aircraft 
at  Eglin  Air  Force  Base. 

Displays  write-ups,  sortie-by-sortie,  to  give  a 
snapshot  history  of  each  aircraft.  Program 
computes  write-up  rates  for  each  aircraft  and 
prints  them  at  the  end  of  each  tail  number. 


Spread-field  list  Provides  a  listing  with  each  field  isolated 

from  the  adjacent  one  by  blanks.  Much  easier 
to  read  than  an  80-80  listing. 

Support  general  codes  Scheduled  maintenance  codes. 

Supply  1050  system  The  1050  is  the  standard  supply  computer. 

System  repeat  write-up  analysis  A  repeat  write-up  is  identical  to  the  write-up 

on  the  previous  sortie. 

Tack  Down  An  exercise  with  C-130  aircraft  at  Pope  Air 

Force  Base. 


Throughput  time 
Tough  data 
TMS 


Time  it  takes  to  get  a  job  out  of  the  computer, 
measured  from  request  to  delivery. 

Data  collected  by  many  persons  at  many  points, 
tough  to  get.. 

Type,  model  and  series  of  aircraft. 


-$vi- 


Turnaround  data 


Output  that  includes  all  maintenance  actions. 


Units  produced 
Vector 

When  discovered  code 

Work  Center  Display 
Work  unit  codes  analysis 

Write-ups 

Z-score 

ZI 

Zone  punch 


A  count  of  maintenance  actions.  Each  job  is 
assigned  one  unit  of  work. 

A  record  that  describes  t he  status  of  a  system 
at  a  given  time. 

Code  showing  when  the  malfunction  is  discovered: 
before  flight,  during  flight,  during  inspection, 
etc. 

Also  called  work  center  queue  sort.  Shows  24- 
hour  pictorial  history  of  work  center. 

Summarizes  on  one  page  all  the  meaningful 
information  on  form  300  records  for  each  work 
unit  code. 

A  malfunction  is  "wr it ter.-up , "  i.e.  ,  described. 
Along  with  break-rates,  write-ups  are  the  major 
independent  variables  in  determining  aircraft 
recovery  and  turnaround. 

Score  expressed  in  sigma  units. 

Zone  of  Interior  (USA) 

11-punch  (-)  or  12-punch  (+)  on  card  used  when 
punching  Alpha  or  special  characters. 
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I.  INTRODUCTION 


Most  maintenance  analyses  involve  two  kinds  of  variables:  inde¬ 
pendent  variables,  which  may  affect  system  behavior;  and  dependent 
variables,  which  may  be  affected  by  system  behavior.  Table  1  lists 
some  variables  that  are  often  used  in  maintenance  analysis. 


Table  1 

MAINTENANCE  ANALYSIS  VARIABLES 


Dependent 


Independent 


Hung  ordnance 

Aborts,  air  and  ground 

Maintenance  nondelivery 

Cancellations 

Late  takeoffs 

Write-up  rates 

Break-rates 

Demand  rates,  supply 

Demand  rates,  specialists 

Demand  rates,  AGE 

Aircraft  turnaround,  net  and  gros  • 

Recovery,  net  and  gross 

Configuration  times 

Man-hours 

Sortie  length 

Lags 

De lays 

Phased  inspections 
Base  self-sufficiency 
Cannibalizat ion 
NORS-G 


Test  periods 

Squadrons- -wings 

Weather  conditions 

Flying  schedules 

Theater  conditions  versus  ZI 

Base  dispersal 

Centralized-decentralized 

Target  difficulty 

Flight  profiles 

Mission  types 

Mission  length 

Experience  length 

Modification  effects 

Tail  number  differences 

Time--age 

Training  effects 

Procedure  changes 


Operationally  ready  rates 
Mission  success 
Manpower  utilization  rates 
AGE  utilization  rates 

In  the  following  pages,  the  examples  show  how  to  analyze  the 
behavior  of  most  of  the  dependent  variables,  but  the  list  of  independent 
variables  is  addressed  mostly  by  implication.  This  is  done  because  the 
list  of  dependent  variables  remains  fairly  constant,  and  these  variables 
are  the  topics  that  maintenance  management  is  most  concerned  about.  The 
independent  variables  change,  depending  on  the  type  of  test  or  exercise 
and  the  use  of  the  weapon  system. 
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In  using  computers  to  assist  in  maintenance  analysis,  our  basic 
data  element  is  a  record  that  originates  as  a  keypunched  card.  The 
record  always  contains  at  least  one  dependent  and  one  independent  vari¬ 
able.  The  dependent  variable  is  used  in  the  calculations,  and  the 
independent  variable  is  a  computer  control. 

Thus  the  following  sets  of  write-ups  might  occur: 

Dependent  Variable  Independent  Variable 
Write-up  Count  Tail  Number 

5  7444 

2  7444 

1  7444 

2  7448 

4  7448 

7  7448 

The  computer  senses  two  fields  in  each  card:  the  write-up,  which  it 
adds  (accumulates) ,  and  the  tail  number.  When  it  senses  a  change  in 
the  tail  number  field,  most  often  it  prints  a  summary,  zeros  out  the 
accumulator,  and  begins  processing  the  second  tail  number,  and  so  on. 

To  do  this  with  a  computer,  the  data  must  be  sorted  so  that  all  like 
tall  numbers  are  together. 

This,  in  capsule  form,  is  the  essence  of  computerized  analysls-- 
sort,  sense,  and  calculate.  Unfortunately,  this  simplicity  rarely 
occurs.  Inspection  of  the  various  card  formats  shows  that  the  typical 
record  contains  about  20  fields.  As  Independent  variables  are  identi¬ 
fied,  they  are  added  to  the  records.  To  keep  the  data  volume  to  man¬ 
ageable  proportions,  therefore,  it  is  impossible  to  tolerate  records 
containing  only  two  entries. 

There  is  a  special  form  of  multiple  record  format  known  as  a 
vector,  which  is  a  record  that  describes  the  state  of  a  system  at  a 
given  time.  A  common  vector  is  the  dally  susmmry  analysis,  which  con¬ 
tains  the  following  entries:  the  number  of  sorties  scheduled  (or 
fragged),  the  number  flown,  the  counts  of  the  various  deviations,  the 
NORS-C  percentages,  the  number  and  kinds  of  targets  struck,  and  so  on. 
There  are  also  entries  for  the  independent  variables:  test  periods, 
weather  conditions,  target  difficulty,  pilot  experience,  and  ground- 
fire  intensity,  to  name  a  few. 
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A  vector  of  major  Interest  to  maintenance  analysis  describes  the 
individual  sorties  and  includes  all  pertinent  maintenance  information 
concerned  with  aircraf:  recovery  from  the  sortie.  The  complete  vector 
Includes  the  operations  information  (where  did  it  go,  what  did  it  do, 
what  did  it  carry,  what  was  its  success);  the  maintenance  information 
(what  equipment  was  affected,  what  types  of  personnel  were  required  to 
recover  the  aircraft;  and  the  supply  information  (what  kinds  and  how 
many  parts  were  required).  As  with  the  previous  vector,  the  associated 
independent  variable  codes  are  included  to  identify  the  circumstances 
that  may  be  affecting  the  data. 

A  small  number  of  independent  variables  can  result  in  a  large  num¬ 
ber  of  data  categories  because  of  the  multiplication  factor.  For  ex¬ 
ample,  the  analyst  might  collect  data  in  two  modes,  centralized  or 
dispersed,  and  each  mode  mi«ht  be  divided  <r.to  d°y  and  night  missions 
of  three  types--escort,  strikes,  and  armed  reconnaissance.  This  results 
in  12  unique  categories  (two  modes  x  two  clock  times  x  three  sortie 
types).  The  number  of  analyses  that  might  be  made  from  this  can  be 
horrendous.  With  the  simplest  of  tests,  one  can  check  the  effects  of 
each  of  the  three  main  variables  (mode,  clock-hour,  and  sortie  type) 
and  the  effects  of  each  of  the  12  composite  categories. 

Assume  the  analyst  has  decided  to  sort  the  above  sequence  as 
follows:  major  on  mode,  intermediate  on  clock-hour,  and  minor  on 
sortie  type.  The  sorting  sequence  is  always  from  lowest  (minor)  to 
highest  (major);  hence  the  minor  pass  would  separate  the  data  Into 
three  groups; 

Minor  »  escort,  strike  and  recce. 

The  Intermediate  pass  would  divide  these  three  into  two  groups: 

Day  *  strike,  escort,  recce 
Night  ■  strike,  escort,  recce. 

The  major  sort  further  divides  the  data  into  two  more  groups: 

Centralized  ■  day  ■  escort,  strike,  recce 
Centralized  •  night  ■  escort,  strike,  recce 
Dispersed  -  day  ■  escort,  strike,  recce 
Dispersed  ■  night  •  escort,  strike,  recce. 
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Assuming  the  sort  was  alphabetical,  the  various  categories  would 
appear  in  the  following  sequence  in  the  deck  if  the  codes  were  the 


first  letter 

of  the 

category. 

1 

COE 

Centralized,  Day,  Escort 

2 

CDR 

Recce 

3 

CDS 

Strike 

4 

CNE 

Centralized,  Night,  Escort 

5 

CNR 

Recce 

6 

CNS 

Strike 

7 

DDE 

Dispersed,  Day,  Escort 

8 

DDR 

Recce 

9 

DDS 

Strike 

10 

DNE 

Dispersed,  Night,  Escort 

11 

DNR 

Recce 

12 

DNS 

Strike 

If  sorting  and  controlling  were  limited  to  this  th ree-chaiacter  field, 

the  computer 

would 

output  12  summaries,  one  for  each  category,  plus  an 

additional  summary  for  the  entire  sample. 

This  seemingly  inordinate  amount  of  discussion  of  sorting  and 
controlling  is  intentional.  Until  the  analyst  can  mentally  sort  end 
resort  data,  all  the  while  visualizing  how  the  computer  output  will 
look,  he  will  only  be  able  to  analyze  at  the  most  pedestrian  level. 

The  best  bet  for  the  neophyte  is  to  obtain  a  small  deck  (say  30  cards) 
and  set  up  a  major,  a  minor,  and  an  80-80  list  for  each  pass  on  the 
card  sorter.  To  do  this  he  must  understand  the  following  sorting 
directions : 

Sort  Major  Cols.  35-38  N  tail  number 
Minor  Cols.  11-12  N  month 

The  N  indicates  a  numeric  sort.  The  sorting  sequence  is  from  lowest 
to  highest  order;  hence  the  first  sort  is  the  lowest  minor  position, 
Col.  12.  The  sequence  of  the  actual  aort  is 

Major  Cols.  12,  11 

Minor  Cols.  38,  37,  36,  35 

Try  this  listing  on  each  sorting  pass,  then  reverse  the  major  and 
minor,  noting  the  effect  on  the  sequencing.  See  Fig.  1. 

The  sorting-sensing-procesaing  aequence  that  is  the  heart  of  an 
analysis  is  always  preceded  by  a  data  selection-tagging  sequence.  It 
is  necessary,  however,  to  have  some  understanding  of  the  what's  and 
why ' s . 
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MINOR  MAJOR  MAJOR  MINOR 


05 

1620 

06 

1620 

07 

1620 

08 

1620 

09 

1620 

05 

1621 

06 

1621 

07 

1621 

08 

1621 

09 

1621 

05 

1622 

06 

1622 

07 

1622 

08 

1622 

09 

1622 

05 

1623 

06 

1623 

07 

1623 

08 

1623 

09 

1623 

05 

1624 

06 

1624 

07 

1624 

08 

1624 

09 

1624 

05 

1625 

06 

1625 

07 

1625 

08 

1625 

09 

1625 

05 

1626 

06 

1626 

07 

1626 

08 

1626 

09 

!626 

05 

1627 

06 

1677 

07 

1627 

08 

1627 

09 

1627 

05 

1629 

06 

1629 

07 

1629 

08 

1629 

09 

1629 

05 

1620 

05 

1621 

05 

1622 

05 

1623 

05 

1624 

05 

1625 

05 

1626 

05 

1627 

05 

1629 

06 

1620 

06 

1621 

06 

1622 

06 

1623 

06 

1624 

06 

1625 

06 

1626 

06 

1627 

06 

1629 

07 

1620 

07 

1621 

07 

1622 

07 

1623 

07 

1624 

07 

1625 

07 

1626 

07 

1627 

07 

1629 

08 

1620 

08 

1621 

08 

1622 

08 

1623 

08 

1624 

08 

1625 

08 

1626 

08 

1627 

08 

1629 

09 

1620 

09 

1621 

09 

1622 

09 

1623 

09 

1624 

09 

1625 

09 

1626 

09 

1A?7 

09 

1629 

K  i  g.  1  --  Sorting  i  1  Inst  r.it  ion 


(This  shows  elicits  oi  reversing  the  ".ajar  and  minor.  Note 
each  major  entry,  the  minor  appear'  In  ascending  order, 
also  he  done  in  iieseending  or  n  ;  if  so  desired.) 
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The  selection  process  is  necessitated  by  the  multi-formatted  data 
files.  The  first  of  these  is  the  master  file  of  all  the  Oxnard  formats-- 
approximately  a  dozen  different  record  formats  that  all  have  (for  sorting 
purposes)  the  tail  numbers  and  start  times  in  a  common  location.  Most 
analyses  require  only  one  of  these  formats.  For  example,  deviation 
analysis  is  done  by  selecting  the  data  with  an  H  in  Col.  80.  The  in¬ 
structions  to  the  computer  room  are  "Select  data  having  an  H  in  Col. 

80".  The  computer  operators  decide  the  appropriate  software  and  pro¬ 
cedures.  Often  a  multiple  selection  is  required.  "Select  data  naving 
a  zero  in  Col.  80  and  731  in  Cols.  53-55".  These  are  the  combat  damage 
records.  The  "and"  is  critical--it  says  that  all  records  must  meet 
both  conditions:  (a)  they  must  have  a  zero  in  Col.  80,  and  (b)  they 
must  also  have  a  731  in  Cols.  53-55.  This  instruction  is  entirely 
different  from  "Select  da^a  having  a  zero  in  Col.  80  or  7''1  in  Cols. 
53-55".* 

Another  way  of  selecting  data  is  to  use  the  matching  record.  A 
typical  instance  would  be  when  we  wish  to  select  the  particular  subset 
of  mission  information  that  matches  the  sortie  records  (Col.  80-2). 

Both  have  the  dates  and  the  call  signs  on  which  the  match  is  to  be 
made.  The  instructions  to  the  computer  operators  are  "Select  the 
mission  records  (Col.  80-Z)  that  match  the  dates  (Cols.  9-12)  and 
call  signs  (Cols.  56-61)  of  the  Col.  80-2  records". 

The  tagging  (identification)  is  somewhat  simpler;  decide  the 
codes  and  the  columns.  When  possible  use  mnemonic  codes,  since  this 
greatly  facilitates  the  Identification  of  the  output:  H  for  hard,  E 
for  easy,  for  example.  See  Fig.  2. 

The  more  the  analyst  works  with  the  data,  the  more  he  will  be 
persuaded  to  avoid  card  storage.  Rather,  once  the  data  are  edited, 
he  will  keep  them  entirely  on  tape.  One  reason  for  this  is  that  the 
tape  record  is  not  restricted  to  80  columns.  Our  practice  has  been 
to  restrict  the  basic  record  to  80  columns,  but  to  use  a  standard 
132-character  tape  record.  The  extra  52  characters  are  used  for  tag¬ 
ging,  special  sorting,  and  ancillary  computations.  Even  this,  at 
times,  ha  oeen  insufficient.  Hence,  most  of  the  general  purpose 
programs  are  written  to  accept  a  264-character  record. 

a 

It  would  select  any  record  meettn*  either  or  both  conditions. 
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II.  METHODOLOGICAL  EXAMPLES 

AIRCRAFT  TURNAROUND  AND  RECOVERY 

The  person  familiar  with  statistical  methodology  realizes  that 
the  present  AFM  66-1  system  does  not  readily  permit  the  computation  of 
variances,  the  sine  qua  non  of  statistical  manipulation.  A  number  of 
attempts  have  been  made  to  circumvent  this  shortcoming. 

An  early  observation  was  that  if  data  could  he  associated  with 
the  specific  sortie,  the  variances  of  sortie  dates  could  be  obtained. 
Accordingly,  a  number  of  attempts  were  made  to  isolate  data  by  sortie. 
The  majority  of  these  early  attempts  centered  around  the  use  of  air¬ 
craft  hours,  which  are  entered  on  the  AFTO-200  series  documents. 

These  early  explorations  were  facilitated  by  the  particular  data 
sample  available  (SAC  B-52  data  from  Beale  Air  Force  Base),  which  in¬ 
cluded  the  debriefing  data  and,  fortunately,  the  aircraft  hours  as 
well.  Although  it  was  impossible  to  make  a  completely  satisfactory 
by-sortie  separation  of  the  data  (because  aircraft  hour  entries  were 
often  inaccurately  recorded) ,  it  was  possible  to  demonstrate  that  in¬ 
teresting  and  meaningful  analyses  could  be  had  if  the  data  could  be 

* 

identified  by  sortie.  At  that  time  the  simple  solution  seemed  to  be 
to  Include  a  sortie  number  on  all  documents — an  idea  considerably 
easier  to  conceive  than  to  implement.  Hence  the  prospect  of  doing 
by-sortie  retrieval  was,  at  best,  discouraging. 

At  this  time,  Bell  and  Smith  were  exploring  the  area  of  mainte¬ 
nance  management  and  included,  as  a  part  of  their  exercises,  the  use  of 

** 

clock-hour  entries  on  the  AFTO-200  series  document.  Consideration 
of  these  data  1eada  to  another  method  of  isolating  data  by  sortie.  This 
technique  used  mixed  record  formats  sorted  by  time-within-tail-number 
sequence.  Treated  in  this  manner,  the  data  appear  in  a  time-ordered 
sequence  in  which  each  sortie  description  record  (Col.  80«2)  is 


* 

W.  H.  McGlothlin  and  T.  S.  Donaldson,  Trends  in  Aircraft  Mainte¬ 
nance  Requirements.  The  RAND  Corporation,  RM-4049-PR,  June  1964. 

**C,  F.  Bell  and  T.  C.  Smith,  The  Oxnard  Base  Maintenance  Manage- 
■ent  Improvement  Program,  The  RAND  Corpo  ation,  RM-3370-PR,  November 

I¥62T^ 
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followed  by  a  series  of  maintenance  records  (Col.  80“0)  which  result 
from  the  aircraft  recovery.  Given  this,  it  ia  possible  to  program  a 
computer  to  summarize  the  data  by  sortie.  On  sensing  the  sortie  de¬ 
scription  record,  the  computer  "initializes"  (clears  ail  accumulators 
and  records  the  contents  of  the  sortie  record).  The  computer  then 
sweeps  through  the  maintenance  data,  record  by  record,  making  the  nec¬ 
essary  calculations.  On  sensing  the  second  sortie  description  record, 
the  computer  outputs  the  accumulations  to  this  point,  initializes,  and 
starts  accumulating  the  summaries  of  the  second  sortie,  and  so  on. 

Our  sortie-summary  program,  known  as  Clint  (Vol.  II),  produces 
three  different  records:  a  summary  of  the  entire  sortie;  a  summary  of 
each  two-digit  system  Involved  ir.  the  aircraft's  recovery;  and  a  sum¬ 
mary  of  each  work  center  that  participates  in  the  recovery.  Each  rec¬ 
ord  is  identified  to  the  sortie  by  including  three  freouently  used 
variables:  the  tail-number,  the  touchdown  time,  and  the  type  of  sortie 
code. 

The  time-sequence  method  of  by-sortie  retrieval  has  one  short¬ 
coming  that  shows  up  in  certain  analyses:  Any  deferred  maintenance  is 

associated  with  the  sortie  where  the  maintenance  occurred,  and  not 

* 

with  the  sortie  on  which  the  write-up  appeared.  Consequently,  those 
analyses  concerned  with  determining  the  effects  of  sortie  type  are 
"contaminated"  hy  including  deferred  items.  As  a  general  statement, 
this  is  not  a  serious  problem  for  two  reasons.  First,  the  relationship 
of  the  deferral*,  to  the  sortie  type  tends  to  occur  on  a  random  basis, 
hence  the  random  variance  is  generally  increased  (which  can  be  circum¬ 
vented  by  increasing  the  sample  size).  Second,  the  deferred  items,  by 
definition,  are  considerably  less  important  than  the  mission  essential 
items.  An  item  must  be  noncritical  or  it  cannot  be  deferred:  for 
example,  the  deferrals  are  items  such  as  bunged  DZU  fittings,  frayed 


This  deficiency  may  be  avoided  by  collecting  when-discovered 
clock  times  to  permit  the  maintenance  requirement  to  be  identified 
with  the  sortie  it  was  most  nearly  associated  with.  Of  course,  such 
associations  do  not  prove  relationships  or  cause  and  effect;  but  they 
represent  useful  approximations.  It  is  interesting  to  note  that 
collection  of  such  information  was  included  in  the  original  Oxnard 
data  system  previously  referenced. 
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cable  lacings,  missing  bolt  plates,  and  so  on.  Indeed,  they  are  often 
referred  to  as  "a  bunch  of  junk." 

Four  sets  of  records  that  Clint  produces  have  proved  most  useful. 

Our  practice  has  been  to  keep  them  separated  on  each  of  four  tapes. 

1.  The  aircraft  (7-cards)  recovery  records  (format  Rl).  These 
include  only  the  unscheduled  maintenance  actions  and  the 
post  flights. 

2.  The  aircraft  (7-cards)  turnaround  records  (format  Rl) . 

These  Include  both  scheduled  and  unscheduled  maintenance 
data. 

3.  The  system  (8-cards)  records  (format  R2)  that  are  produced 
concurrently  with  the  aircraft  turnaround  records.  These 
Include  both  the  scheduled  and  unscheduled  maintenance 
actions. 

4.  The  work  center  (9-card)  records  (format  R3)  that  are  the 
analog  of  the  system  records.  These  Include  the  actions 
of  all  of  the  work  centers,  scheduled  and  unscheduled 
maintenance. 

The  sortie  data  when  output  on  tape  (almost  Inevitably  the  case) 
is  a  record  containing  264  characters.  The  sortie  write-up  count 
appears  in  Cols.  131-.  12,  and  the  sortie  break-rate  data  appear  in 
Col.  130  (blank  if  no  write-ups  and  1  if  one  or  more  write-ups  occur). 

The  system  breaks  (coded  blank  or  1)  recorded  in  Cols.  94  through  129 
show  the  number  of  systems  used. 

Most  analyses  of  the  turnaround  and  recovery  data  will  compare 
various  periods  or  test  phases.  Providing  that  no  statistical  testing 
is  desired,  the  quickest  and  most  communicable  way  to  obtain  a  com¬ 
plete  data  summary  is  to  process  the  7-card  records  with  the  Recovery 
Summary  program  (RECSUM,  also  called  the  Aircraft  Histogram).  If  the 
unit  is  flying  mostly  daylight  sortlas,  the  most  useful  display  will 
be  obtained  by  majoring  on  the  squadron  designation  (Col.  1)  and  minorlng 
on  single,  first,  and  last  sorties  (Col.  21).  The  histogram  entries 
most  generally  used  are  the  Col.  21  entries  (SFL),  the  first  digit  of 
the  sortie  type  code  (Col.  17),  or  the  touchdown  time  code  (Col.  16). 

Net  fix  time  (Cols.  38-42)  is  generally  used  as  the  variable  detailed 
in  the  matrix  entries.  See  Fig.  3. 


PAJOH —  !  t'l'>  l-  1  COPPAAISON  0*  SC«TIF  TYPES  KH  TUPKAKOUSO  TAPE 

*  Cf'LS  ti-  71 

Sl'Tr.U.  LAST  AMO  All  C?HF  H  SCftllES  SOA  TIE  TVP* 


I 


o  o 
^  o  o  o 
uoocooo^ 
u  u  u 
c*  O  O  O 
CO* 

c 

o?cooauoo 
o  o  u 
_■  o  o  c  a-  ?  o 
ooc 
0000321 
t>  o  o  o 

aaooooooooso 

oooooe 

OCCUOC2002X 

o  5  o 

aoocoooox 

oooooooo  88  8  i 

OC-3gOo2cS§ 
uo*o 3oa; 
oaoooooocoi 

C.QQI 

o^oocacoeoaoocccjcpora 

e  o  o  c  a  o 
oo-iooooocc 
o  o  o  u  a  o  c 

°S8d88**5§i 

oazzociocoacccoaaaucooouQoc 
acoousoaSoca  o  c  ~  c  opr 


accioooaoeoaaoooao 
o  e  o  o  g 


w  C  yO 

«J  U  P- 

a  ^  — 


a  «*  *  rs 

u.  u  — 

—  %  < 

a  -> 

w*  x 
*  i 
a  u 

CO  -If 

I  •  •  *- 


«-fc---.=w«yCCG.  -  t*  C.  C  Z  - 

-3-jjii.oiSi  35*5 

o3Lo::.ooc.w::  .2  o 

■-  -  i:  — -  ■ 


«  i»  *  <»  i>  -• 


\NOT  REPRODUCIBLE 


-12- 


In  most  Instances  a  statistical  comparison  is  desired.  This  will 
generally  be  done  using  the  Analysis  of  Variance  program.  Title  (for¬ 
mat  C21)  and  a  control  card  (format  C2)  must  be  made  for  each  data 
field  to  be  sensed.  Since  the  sorting  for  RECSUM  and  Analysis  of 
Variance  is  the  same,  our  practice  has  been  to  obtain  both  sets  of 
outputs  while  the  data  are  so  sorted.  (The  big  element  affecting 
throughput  time  is  sorting,  which  generally  takes  five  to  ten  times 
the  amount  of  time  that  either  of  the  two  programs  takes.) 

In  making  up  the  title  and  control  cards  for  Analysis  of  Variance, 
it  is  easier  to  layout  and  keypunch  both  if  the  common  types  of  in¬ 
struction  are  held  separate.  This  is  shown  in  the  following. 

TITLE  CARD  LAYOUT 

Pass  Aircraft  Turnaround  by  Test  Period  Variable 

1  lags 

2  delays 

3  net  fix 

4  gross  fix 

5  units 

6  man-hours 

7  sortie  length 

CONTROL  CARD  LAYOUT 


Sort  Control 

Maj  1-1  Maj  1-1 
Pass  Min  74-77  Min  74-77 


Variable 


1  28-32  lags 

2  33-37  delays 

3  38-42  net 

4  37-47  gross 

3  58-61  units 

6  61-63  man-hours 

7  66-68  sortie  length 

NOTE:  Switches  A,  B,  and  D  are  "on"  for  all 

passes. 


The  arrows  are  keypunch  Instructions  equivalent  to  ditto  marks.  The 
special  Instructions  to  the  computer  room  always  describe  the  spaclfic 
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data  bank  (or  type)  to  selert.  The  third  pass,  net  turnaround,  is 
shown  in  Fig.  4. 

As  discussed  in  the  Analysis  of  Variance  section,  the  analyst  may 
also  wish  to  determine  the  impact  of  NORS-G  on  turnaround  and  recovery — 
and  also,  particularly  when  dealing  with  complex  aircraft  such  as  the 
F-4  and  the  B-S2,  to  determine  the  effect  of  the  phased  inspections. 

Gross  turnaround  and/or  gross  recovery  figures  are  the  most  sensi¬ 
tive  barometers  to  program  changes;  as  the  flying  schedules  increase, 
the  turnaround  times  decrease  and  vice  versa.  These  changes  will  show 
mostly  in  the  delay  portion  of  the  turnaround.  This  is  evidenced  by 
the  high,  positive  correlations  between  gross  turnaround  and  delay 
entries.  Pearson  product  moment  correlations  normally  range  from  0.90 
to  0.95.  In  contrast,  program  changes  have  little  impact  on  the  net 
turnaround  and/or  net  recovery  data,  although  some  reductions  can  b’ 
expected.  By  and  large  the  mechanism  for  dealing  with  an  increased 
flying  program  does  not  so  much  concern  speeding  up  the  turnaround, 
rather  it  insures  that  aircraft  are  not  sitting  idle.  As  would  be  ex¬ 
pected,  just  as  there  is  little  change  in  the  net  turnaround  data, 
there  is  also  little  change  in  the  man-hour  data  when  the  program  is 
altered.  This  is  also  shown  by  the  correlations  between  net  fix  times 
and  man-hours,  which  also  tend  to  be  high  and  positive — 0.90  to  0.95. 

Some  sortie  types  are  definitely  harder  on  both  aircraft  and 
pilots,  and  all  of  the  maintenance  measures  usually  reflect  this.  The 
hard  sorties  result  in  more  write-ups  and  hence  require  more  fix  ac¬ 
tions.  The  high  write-up  rates  result  in  Increased  man-hour  consump¬ 
tion,  Increased  units  produced,  Increased  turnaround  times,  and  so  on. 

In  addition,  the  larger  number  of  Jobs  lncreasss  the  exposure  to  delay 
potential,  hence  aircraft  delays  also  increase. 

In  contrast  to  the  sortie  type  effect,  the  sortie  length,  per  sc, 
has  little  or  no  effect.  In  only  one  analysis  (of  several  score)  have 
we  seen  a  sortie  length  effect,  and  this  was  minor.  It  occurred  in  a 
study  comparing  B-52  24-hour  missions  with  B-52  8-hour  missions;  a  17- 
percent  increase  in  man-hour  consumption  characterised  the  longer 
missions.  Even  in  this  instance,  it  could  not  be  determined  unequivo¬ 
cally  that  this  t.as  not  a  sortie  type  effect. 
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A  fairly  good  estimate  of  how  well  a  unit  might  do  in  an  acceler¬ 
ated  effort  can  be  had  by  reviewing  data  on  the  first  sorties  of  the 
day.  These  are  the  missions  for  which  the  aircraft  has  been  turned 
around  quickly  because  it  is  programmed  to  fly  again  the  same  day. 

These  turnarounds  consist  mostly  (.but  not  entirely)  of  mission-essen¬ 
tial  maintenance  actions. 

One  thing  the  analyst  will  quickly  discover  is  another  Parkinson's 
law:  an  aircraft  takes  as  long  to  turn  around  as  there  is  time  avail¬ 
able.  This  finding  always  engenders  a  feeling  of  dismay.  On  a  moment's 
reflection,  however,  the  analyst  will  recall  that  the  turnaround  data 
Include  all  the  launch  actions  (towing,  top-up,  final  checkout,  for 
example)  that  occur  just  before  takeoff.  Thus  the  turnaround  data 
should  be  (and  are)  a  fairly  close  representation  of  the  flying  sched¬ 
ule.  This  is  why  the  net  turnaround  and  delay  data  are  important; 
they  contain  the  germinal  determinations  of  what  could  be  done  under 
stress. 

When  processing  the  aircraft  recovery  (as  opposed  to  turnaround) 
data,  one  needs  to  remember  that  the  recovery  delay  summaries  consist 
of  two  elements:  true  delays  resulting  from  ;.o  maintenance  actions, 
and  apparent  delays  resulting  from  the  ''holes"  left  by  removing  the 
scheduled  maintenance  data.  The  difference  between  the  two  shows  the 
amount  of  scheduled  maintenance  that  is  not  done  concurrently  with  un¬ 
scheduled  maintenance.  Where  aircraft  utilisation  must  be  Increased, 
one  does  not  like  to  see  this  difference  appreciably  exceed  the  com¬ 
bined  time  to  fuel  and  upload  the  aircraft  (neither  action  can  be  done 
concurrently  with  other  maintenance),  since  this  would  indicate  that 
there  is  not  as  much  concurrent  maintenance  as  might  be  desired. 

WRITE-UPS  AMD  BREAK-RATES 

Of  all  the  measures  available  to  help  understand  and  control  air¬ 
craft  maintenance,  write-ups  and  break-rates  are  most  Important. 

These  are  the  major  independent  variables  in  determining  aircraft  re¬ 
covery  and  turnaround,  and  therefore  the  major  determiners  of  sortie 
generation  capability.  Break-rates  also  determine  the  damand  rates 
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for  Job  specialists,  AGE,  and  supply,  and  hence  are  of  prime  impor¬ 
tance  in  solving  resource  allocation  problems.  As  will  be  shown,  the 
break-rates  used  in  conjunction  with  the  job  duration  data  provide 
the  bulk  of  information  needed  to  comprehend  sortie  generation  capa¬ 
bility  and  resource  needs.  Since  job  durations  tend  to  be  stable,  our 

A 

major  consideration  is  to  understand  the  break-rate  data. 

All  unscheduled  maintenance  begins  with  a  write-up  In  the  781A 
folder.  Thus  if  the  flight  or  the  ground  crews  diacover  anything 
aberrant,  they  enter  it  (in  English)  in  the  781  jacket.  The  781A  is 
organized  in  pairs  of  blocks.  When  a  discrepancy  is  discovered,  its 
characteristics  are  entered  in  a  left-hand  block;  when  the  discrepancy 
is  corrected  (cleared),  the  correction  is  described  in  the  correspond¬ 
ing  right-hand  block.  The  details  of  the  correction  are  also  entered 
in  the  associated  AFTO  200  series  forms — the  form  300  of  the  RAND/TAC 
system.  The  300  forms  are  then  keypunched  and  serve  as  the  basis  of 
the  write-up  and  break-rate  computations. 

In  conjunction  with  the  AFTO  200  series  recording  system,  the 
Air  Force  Logistics  Command  publishes  a  manual  (called  the  06  manual) 
for  each  type,  model  and  series  (IMS)  of  aircraft.  The  -06  manuals 
describe  the  aircraft  broken  out  into  5-digit  work  unit  codes;  for 
example,  111AB  equals  Radome  zipper  seal,  and  511BE  equals  true  air 
speed  indicator.  The  first  two  digits  are  used  in  the  write-up  and 
break-rate  computations.  These  two-digit  break-outs  represent  the 
major  aircraft  systems:  11  airframe,  12  cockpit  and  fuselage,  13 
landing  gear,  14  flight  controls,  23  propulsion  system,  and  so  on. 

The  Clint  program  dees  the  bulk  of  the  write-up  computation 
using  a  table  of  2-diglt  system  codes  to  control  processing.  As  each 
maintenance  record  la  read  in,  the  first  two  digits  of  the  work  unit 
code  ere  compared  with  the  system  table.  If  a  match  is  found,  the 
suaauirles  against  that  system  are  initiated.  This  continues  until  all 
data  for  that  particular  sortie  are  processed  (indicated  when  the  com¬ 
puter  encounters  the  next  sortie  descriptor  card).  At  this  point,  the 


T.  S.  Donaldson  and  A.  F.  Sweet  land.  Trends  in  F-101  Maintenance 
Requirements,  The  RA1VD  Corporation,  RM-4930-PR,  April  1966. 
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computer  produces  a  summary  record  for  each  2-digit  system  that  has 
required  maintenance — one  record  for  each  system.  (To  reduce  data 
volume,  no  record  is  produced  for  systems  showing  no  maintenance.) 

Given  these  system  records  (generally  referred  to  as  8  cards 
because  they  have  an  8  in  Col.  2),  the  system  break-rates  are  computed 
by  dividing  each  system's  record  count  by  the  number  of  sorties.  The 
aircraft  write-ups  per  sortie  are  obtained  by  dividing  the  total  num¬ 
ber  of  system  records  by  the  sortie  count. 

To  facilitate  this  process,  the  Analysis  of  Variance  program  has 
been  modified  to  perform  all  the  necessary  calculations.  The  instruc¬ 
tions  to  the  computer  operator  are  "Select  the  Clint  output  8-cards 
(format  R2)  having  entries  greater  than  zero  in  Col.  23  (zero  entries 
in  Col.  23  are  the  support  general  data),  sort  and  control  as  follows" 

Major  Col.  1  AN  squadron 

Minor  Cola.  23-24  N  2-digit  system 

Variable  Cols.  38-42  N  net  fix  time 

The  appropriate  sortie  count  is  entered  in  Cola.  1-4  of  the  control 
card  (format  C2) .  The  write-up  rate  sppears  at  the  bottom  of  the 
break-rate  field.  See  Fig.  5. 

Switch  B — "off"  to  produce  the  system  break-rates. 

Switch  G--"off"  until  all  input  data  have  been  read  in. 

"on"  allows  the  user  to  proceed  from  this  point. 

Aircraft  break-rate  la  another  measure  obtainable  from  the  data. 
This  Is  defined  as  the  number  of  sorties  having  at  least  one  write-up, 
divided  by  the  total  number  of  sorties.  Note  that  the  scoring  Is 
dichotomous;  the  aircraft  la  "broken"  (regardless  of  the  number  of 
write-ups)  or  "well."  Clint  produces  this  count.  When  the  Clint 
program  is  output  on  tape,  a  complete  record  of  system  and  aircraft 
write-ups  Is  kept  on  the  sortie  (Col.  2-7)  (format  Rl)  tape  record, 
which  Is  264  characters.  Columns  94-129  are  set  aside  to  record 
system  breaks;  for  example,  if  system  11  requires  unscheduled  mainte¬ 
nance,  set  Col.  94  equal  to  1,  otherwise  set  to  blank.  Just  before 
output,  the  computer  sweeps  Cols.  94-129  counting  the  breaks.  This 
count  is  entered  in  Cols.  131-132.  If  the  write-up  count  Is  greater 
than  zero,  a  1  Is  entered  in  Col.  130  Indicating  a  broken  sortie.  If 
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(The  last  column  on  the  right  gives  the  system  break-rates  (reliability). 
The  system  fix-times  (maintainability)  are  shown  in  the  "means"  column 
The  F-test  is  not  appropriate  for  this  sample  and  is  ignored.) 
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no  write-ups  have  occurred,  the  column  is  left  blank.  When  these  tape 
records  are  processed  using  the  aircraft  histogram  program,  KECSUM 
(Vol.  II),  the  aircraft  write-up  and  break-rate  computations  (totals 
and  averages)  appear  automatically  at  the  bottom  of  the  page. 

The  aircraft  write-ups  and  break-rates  can  also  be  obtained  by 
using  the  Frequency  Counter  (FREQ)  (Vol.  II).  In  this  instance,  the 
appropriate  system  records  (Cols.  ?-8  and  Col.  23  have  only  entries 
greater  than  zero)  are  selected.  The  data  are  sorted  in  Cels.  3-15, 
which  allows  all  data  for  each  sortie  to  be  placed  together.  FREQ 
senses  this  same  field.  The  total  count  is  the  number  of  write-ups; 
the  line-counts  are  equal  to  the  number  of  broken  sorties.  Dividing 
these  two  counts  by  the  sortie  count  gives,  respectively,  the  aircraft 
write-ups  per  sortie  and  the  aircraft  break-rates  per  sortie. 

In  general,  write-up  data  are  preferable  to  break-rate  data. 

This  is  because  the  latter  *•  w  away  much  information  by  ignoring  the 
number  of  write-ups .  To  show  the  relationships  between  write-upa  and 
a  number  of  the  conventional  maintenance  measures,  the  following  table 
is  offered.  The  measures  are  based  on  aircraft  recovery,  not  turn¬ 
around.  Thus  all  support  general  actions  except  the  poBtflights  have 
been  removed. 


Table  2 

pearson  product  moment  correlations  between  aircraft 

WRITE-UPS  AND  OTHER  MAINTENANCE  MEASURES 


Aircrai  t 

'? . ■"  1 


Measure 

[F-5A  I 

F-100 

F-4C 

Net  recovery 

h  | 

0,633 

0.594 

0.749 

Groaa  recovery 

0.507 

0.431 

0.580 

Units 

0.851  , 

0.8<44 

0.864 

Man-hour  a 

C.  631  | 

0.550 

0.730 

Not  shewn  in  Chest  calculations  are  the  aircraft  delay*.,  which 
correlate  highly  (about  0.95)  with  groas  turnaround.  The  essence  of 
the  information  ir.  Table  2  la  that  a  sortie  with  a  large  number  of 
write-ups  makes  a  difficult  scheduling  problem  for  Job  control. 


-20- 


Frequent  ■  'ite-up  races  cause  long  turnarounds  as  well  as  Increased 
demands  for  manpower  and,  presumably,  other  resources. 

Of  vital  Interest,  of  course,  is  what  causes  the  frequency  of 
write-ups.  One  factor  is  aircraft  characteristics.  Each  TMS  has  its 
unique  reliability  characteristics.  A  complex  aircraft  such  as  the 
F-4C  has  write-up  rates  averaging  2.5  to  3.5  per  aortle.  In  contrast, 
a  simple  aircraft  such  as  the  A-37A  or  the  F-5A  has  low  write-up  rates 
averaging  0.5  to  O.d  per  sortie.  Given  a  specific  TMS,  the  type  of 
sortie  has  s  definite  impact  on  this  basic  break-rate.  When  one  com¬ 
pares  hard  and  easy  missions  flown  by  a  specific  wing  or  squadron, 
substantial  differences  show.  As  an  example,  the  write-ups  on  the 
Rapid  Roger  F-4C  hard  sorties  are  40  to  50  percent  greater  than  those 
on  the  so-called  easy  sorties.  Even  greeter  differences  have  been 
observed  in  the  Z1  data:  navigation  training  missions  produce  twice 
as  many  write-ups  as  guided  air  missile  training  also ions  do. 

To  sumarlze,  then,  since  no  other  single  variable  affects  main¬ 
tenance  behavior  as  much  as  the  write-up,  it  is  the  measure  that 
should  receive  the  most  attention. 

JOB  DURATIONS 

When  analyzing  aircraft  turnaround,  one  must  remember  that  air¬ 
craft  turnaround  is  a  different  phenomenon  from  the  turnaround  of  the 
Individual  write-up.  Aircraft  turnaround  time  i*  almost  entirely  de¬ 
termined  by  the  manner  in  which  job  control  sequences  a  number  of  in¬ 
dividual  scheduled  and  unscheduled  maintenance  actions.  It  is  the 
durations  of  these  individual  actions  that  we  deal  with  in  the  follow¬ 
ing  diacusslon. 

There  are  two  ways  to  compute  job  durations.  From  the  viewpoint 
of  maintenance,  and  particularly  Job  control,  the  Important  consider¬ 
ation  is  the  length  of  the  Individual  actions;  e.g.,  how  long  does  it 
take  to  fix  the  radsr  system.  The  computed  mean  is  often  referred  tc 
as  the  Job  standard;  e.g.,  the  average  fix  time  for  the  radar  system 
la  1.3  hours. 

There  is  another  way  of  computing  the  average  downtime.  Instead 
of  dividing  the  total  downtime  by  the  number  of  fixes,  divide  it  by 
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the  number  of  sorties.  The  means,  which  «re  much  smaller  than  the  job 
standards,  provide  an  immediate  comparison  for  determining  the  systems 
causing  the  most  downtime.  (Th»  same  comparison,  of  course,  can  be 
obtained  from  the  total  downtime  data.)  A  similar  comparison  can  be 
made  by  dividing  the  total  system  downtime  by  the  total  possessed  air¬ 
craft  hours.  Multiplying  this  vesult  by  100  yields  the  percentage  of 
downtime  caused  by  the  system  in  question. 

Both  the  job  standards  computations  and  the  average-per-sortie 
computations  are  provided  by  the  Analysis  of  Variance  program.  If 
switch  B  is  set  "on"  and  the  sortie  count  is  entered  in  Cols.  1-4  of 
the  control  card,  the  job  standard  computations  appear  under  the  "means" 
column  and  the  average-per-sortie  computations  appear  to  the  right  of 
the  variance  computation.  (If  switch  B  la  set  "off",  the  break-rates 
appear  in  the  break-rate  computation  field.)  To  obtain  the  above, 
select  the  8-cards  output  by  Clint  (Col.  2“8) . 

Sort,  control  Major  Col.  1  squadron 

Minor  Cola.  23-24  system 

Variable  Cola.  38-42  net  fix  time 

Switches  A,  B,  D-- "on". 

Switch  G — "off"  until  all  the  input  data  have  been  read  in. 

To  proceed  from  thia  point,  set  G  "on". 

The  F-test  at  the  bottom  of  Fig.  6  applies  only  to  the  means 
columns  (i.e.,  the  Job  standards).  An  F-test  is  not  produced  for  the 
average-per-sortie  means.  We  have  made  no  attempt  to  provide  this 
latter  because  of  the  extremely  peculiar  distribution  of  the  data — a 
tall  column  of  zero  fix  times  (for  those  sorties  having  no  write-ups) 
followed  by  a  tiny  lognormal- like  distribution: 


OmtltUT  (ft'l 


(These  are  the  same  data  as  shown  in  Fig.  5.  The  switch  setting  produces 
the  average  fix  time  based  on  all  sorties  flown,  the  last  field  under  "Breaks 
Average".  In  general  this  computation  is  not  as  meaningful  as  the  entries 
under  "Totals",  which  show  the  total  time  lost  to  maintenance.) 


In  instances  where  we  have  tested  the  average-per-sortie  means,  which 
was  accomplished  by  adding  the  appropriate  number  of  zero  fix-times 
records  (n  ■  sortie  count  -  write-up  counts) ,  the  data  have  pronounced 
inequality  of  variance  (see  Sec.  III). 

Our  experience  has  been  that  the  job  standards  computed  on  net  fix 
times  remain  remarkably  stable  despite  dramatic  changes  in  the  demands 
levied  on  the  system.  For  example,  in  a  recent  exercise  in  Southeast 
Asia,  although  the  sortie  production  rate  was  doubled,  almost  all  the 
system  net  fix  times  (unscheduled  maintenance)  remained  unchanged. 

The  causes  of  the  exceptions  to  this  were  immediately  determinable. 

For  example,  an  influx  of  new  pilots  was  associated  with  an  immediate 
increase  in  the  number  of  write-ups  on  the  fire  control  system  (re¬ 
flecting  the  uneasiness  of  the  new  pilots  in  a  combat  setting).  The 
Job  standards  showed  a  distinct  decrease  in  length,  with  a  high  pro¬ 
portion  of  how  mal  799  (no  defect  discovered)  and  minor  alignments. 

In  two  to  three  weeks,  when  the  pilots  became  adjusted  to  the  combat 
setting,  the  write-up  rates  returned  to  normal  levels  as  did  the  fix 
times.  As  a  general  statement,  changes  in  net  fix  times  almost  always 
Indicate  atypical  or  abnormal  conditions  and  should  be  immediately 
investigated. 

When  computing  the  job  standards,  it  is  also  desirable  to  deter¬ 
mine  the  amount  of  job  delays.  These  delays  ar  caused  by  interrup¬ 
tions  to  the  Job  in  progress;  for  example,  the  item  is  removed  for 
bench  check  or  the  job  is  deferred  because  of  more  pressing  demands. 
Shift  changes  and  lunch  breaks  also  contribute  heavily  to  delays. 

The  data,  which  are  already  sorted,  are  run  through  Analysis  of  Vari¬ 
ance  using  Cols.  33-37  (delays)  as  a  variable  and  again  using  Cols. 
43-47  (gross  fix  times)  as  a  second  variable.  As  indicated  by  the 
following  data,  the  system  delay  tlmss  can  be  substantive.  Note  the 
reduction  in  system  delays  in  test  periods  2  and  7  This  is  almost 
entirely  due  to  management  action. 
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Table  3 

TWO-DIGIT  SYSTEM  DELAY  AVERAGE  IN  HOURS 
(Unscheduled  maintenance  only) 


Test 

Period 

Mean 

Totals 

Sorties 

Flown 

1 

22.02 

36025.1 

454 

2 

5.89 

11941.0 

563 

3 

14.27 

8689.9 

139 

4 

13.10 

43637.9 

844 

5 

15.96 

31461.1 

6 

15.05 

73029.8 

1308 

7 

4.78 

12769.1 

870 

It  is  worth  noting  that  the  amount  of  system  delays  that  Clint 
computes  is  much  greater  (more  than  20  times)  than  those  reported  as 
aircraft  delays.  For  this  reason  we  caution  the  analyst  against  fre¬ 
quently  using  the  delay  codes  in  an  absolute  sense.  Adequate  delay 
reporting  is  often  difficult  to  obtain;  nonetheless  it  does  provide 
useful  information. 


WORK  CENTER  DEMAND  RATES  AND  JOB  DURATIONS 

As  a  general  statement,  the  work  center  data  are  not  of  as  much 
interest  to  the  operational  unit.  This  occurs  partly  because  a  more 
complete  set  of  work  center  analyses  has  been  developed  for  use  at 
the  unit  level  (see  the  section  on  Finding  the  Short  Resource),  and 
partly  because  the  system  break-rate  and  job  duration  analysis  pro¬ 
vides  much  of  the  information  made  available  by  the  work  center  anal¬ 
ysis.  This  occurs  because  most  work  center  actions  are  defined  by 
the  work  unit  codes;  the  engine  shop  works  on  the  power  plants,  the 
radar  shop  on  the  radars,  and  so  on. 

The  processing  of  the  work  center  data  is  identical  to  that  of 
the  break-rate  and  Job  durations  except  that  the  9-cards  of  the  Clint 
output  (Col.  2»9,  format  R3)  are  used,  and  the  minor  sense  is  Cols. 
23-27  (Instead  of  23-24). 


Sort,  control  Major  Col.  1  squadron 

Minor  Cols.  23-27  work  center 

Variable  Cols.  38-42  net  fix 


It  is  recommended  that  the  delays  (Cola.  33-3?)  and  gross  Hx  times 
(Cols.  43-47)  also  be  computed  at  this  time.  As  with  the  8-card  data, 
lag  times  are  not  included  in  the  gross  fix  times. 

When  doing  the  manpower  analysis,  to  aid  in  determining  shift 
allocations,  it  is  helpful  to  compute  the  work  center  lag  times  (lags 
are  measured  from  touchdown  to  the  beginning  of  the  first  job).  On 
doing  this,  the  analyst  often  finds  that  some  work  centers  rarely,  if 
ever,  start  work  until  two  to  three  hours  after  touchdown.  In  these 
Instances,  the  appropriate  reporting  times  can  often  be  adjusted  to 
avoid  the  problems  that  arise  when  a  work  center  has  nothing  to  do  at 
the  beginning  of  a  shift,  but  must  work  overtime  at  the  end. 


TOUCHDOWN 


Net  TA  =  WORK(a)  +  (b)  +  (c) 


WORK 

WORK 

WORK 

(«) 

(b) 

•  •  •  SHIHBB  • 

(c) 

A  data  processing  problem  occasionally  arises  because  of  the 
local  practice  of  redefining  work  centers.  For  example,  a  unit  estab¬ 
lishes  a  special  work  center  to  do  pre-  and  postflights  exclusively. 
After  trying  this  for  a  while,  it  goes  back  to  the  regular  crew-chief 
type  of  operation.  The  result  is  that  the  work  center  demand  rates 
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and  manpower  utilization  data  seem  to  show  sudden  increases  and  de¬ 
creases.  This  makes  the  manpower  distributions  awkward  to  adjust. 

Where  the  work  center  numbering  is  changed  in  the  middle  of  the  job, 
the  fix  is  seen  by  Clint  as  being  two  separate  Jobs.  The  solution  is 
to  recode  the  data  giving  a  common  work  center  code  to  the  segmented  work 
centers.  This  should  be  done  with  the  data  used  for  Clint  input,  not 
the  9  carls  of  the  Clint  output. 

As  previously  mentioned,  the  work  center  analysis  should  not  be 
expected  to  provide  much  information  beyond  that  obtained  from  the 
2-digit  system  analysis  (i.e.,  the  instrument  shop  works  on  instru¬ 
ments).  The  exceptions  to  this  are  the  crew  chiefs  who  do  a  variety 
of  repair  actions  in  addition  to  the  servicing  and  inspections.  Note 
also  that  most  shops  will  record  a  variety  of  support  general  actions 
(e.g. ,  crystal  changes)  as  well  as  a  number  of  special  inspections. 

In  general,  this  is  a  small  part  of  the  workload. 

The  analyst  should  remember  that  the  9-card  analysis  only  includes 
flight-line  actions.  Bench  actions  are  excluded.  Hence  the  9-card 
data  do  not  tell  the  complete  story.  Some  shops  (particularly  the 
avionics  and  engine  shops)  expend  more  than  50  percent  of  their  man¬ 
power  on  shop  actions.  The  off-equipment  utilization  is  obtained  from 
the  normal  AFM  66-1  data  bank  following  AFM  171-14  procedures. 

DIFFERENT  BREAK  AND  RECOVERY  COMPUTATIONS 

While  the  most  useful  computations  are  those  of  the  turnaround 
characteristics,  because  they  give  the  best  picture  of  weapon  ays  ter. 
capability,  they  are  not  the  only  useful  computations.  Indeed,  when 
our  interest  ia  concerned  more  with  hardware  characteristics,  there 
are  a  number  of  better  ways  to  process  the  data.  This  is  shown  in 
the  following. 

Suppose  the  analyst  is  Interested  in  hardware  reliability- 
maintainability.  Then  be  would  probably  not  wish  to  have  the  data 
"contaminated"  with  the  tech  order  compliance  information  indicated 
by  the  following  no  defect,  how  malfunctioned  codes: 
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797  Technical  order  previously  complied  with 

798  Equipment  to  be  replaced,  modified  or  removed 

801  Technical  order  compliance 

802  Technical  order  partial  compliance 

803  Time  change 

804  Removed  for  scheduled  maintenance 

811  Class  modification 

Depending  on  the  particular  question,  all  or  part  of  these  data 
might  better  be  excluded  from  the  computations.  The  analyst  might 
wish  to  exclude  two  other  no  defect,  how  malfunctioned  codes  that  are 
not  related  to  technical  orders.  These  are  code  800  (removed  or  re¬ 
placed  to  facilitate  maintenance)  and  code  799  (no  defect  discovered). 
Code  799  is  used  when  the  specialist's  diagnosis  does  not  confirm  the 
write-up  or  when  he  is  installing  a  good  part. 

While  pursuing  the  hardware  characteristics  area,  our  analyst  may 
also  wish  to  think  about  action  taken  codes  T  (removed  for  cannibali¬ 
zation)  and  l'  (replaced  after  cannibalization).  These  are  both  ac¬ 
tions  on  the  aircraft  being  cannibalized,  and  are  concerned  with  ob¬ 
taining  and  replacing  a  good  unit  for  repairing  a  broken  aircraft. 

In  looking  at  hardware  characteristics,  there  are  times  when  one 
also  wishes  to  look  at  actions  in  more  detail  than  at  the  2-digit 
system  level  (l.e.,  the  hot  section  of  the  power  plant,  uploading  and 
configuring  actions,  preflights,  postflights,  and  so  on).  In  these 
instances,  the  data  must  first  be  recoded.  We  suggest  using  2-digit 
alpha  mnemonics  to  insure  that  the  data  do  not  become  confused  with 
the  regular  data  pool.  The  new  codes  must  be  Included  in  the  table  of 
system  codes  Clint  uses. 

PHASED  AND  HOURLY  INSPECTIONS 

The  phase  sortie  includes  three  types  of  data:  the  regular  data 
resulting  from  the  effects  of  the  sortie,  the  inspection  data  result¬ 
ing  when  the  inspection  crew  works  through  the  inspection  cards,  and 
the  fix  data  detailing  the  repair  of  those  items  found  defective 
during  the  inspections.  The  details  of  the  phase  sortie  are  best  seen 
by  reviewing  one  of  the  flight-line  displays.  Following  touchdown, 
the  maintenance  actions  on  the  display  first  show  the  conventional 
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entries  of  normal  recovery.  The  start  of  the  phase  inspections  is 
signaled  by  a  P  (or  E  for  hourly  inspections)  appearing  in  the  T 
(for  type  of  maintenance)  column.  This  is  taken  from  the  second  char¬ 
acter  of  the  work  order  prefix.  The  phase  inspection  records  can  also 
be  identified  by  the  work  unit  code  which  is  always  0341,  followed  by 
an  alpha  identifying  the  particular  phase  deck  (or  book)  being  used, 
e.g.,  A  for  book  1,  B  for  book  2,  and  so  forth. 

The  fix  actions  that  result  from  the  inspection  are  identified 
in  the  when  discovered  code  (Col.  92)  by  an  M  which  stands  for  discov¬ 
ered  during  special  inspection.  Of  the  many  analyses  that  could  be 
done  on  these  data,  the  following  is  presented  as  the  best  first  pass. 
Make  a  dletrlbution(s)  of  the  turnaround  of  the  phased  sortie  and  com¬ 
pare  it  with  the  dlstrlbutlon(s)  of  only  the  phase  items.  This  is 
detailed  in  the  following. 

Select  the  Clint  output  sortie  cards  from  the  all  systems  data 
(Col.  2"7)  having  a  P  in  Col.  20.  (It  is  best  to  80-80  list  these 
cards  at  this  point  to  find  any  data  errors  by  cross-checking  the 
80-80  listing  with  the  flight-line  displays.)  Using  Analysis  of  Vari¬ 
ance,  obtain  the  net  and  gross  turnaround  distributions  and  also  the 
man-hour  distributions.  In  the  following  example,  the  controls  major 
on  the  squadron  and  minor  on  the  month.  The  three  title  cards,  of 


couree. 

identify 

the  analysis 

and  type 

>  of  data. 

Major  & 

Pass 

Minor 

Sort 

Control 

Variable 

Switches 

"On 

1 

Maj  1-1 

Squadron 

1-1 

38-42  net 

A, 

8, 

D 

Min  8-9 

Month 

8-9 

2 

Maj  1-1 

Squadron 

1-1 

43-47  gross 

A, 

B, 

D 

Min  8-9 

Month 

8-9 

3 

Maj  1-1 

Squadron 

1-1 

61-65  man-hours 

A, 

B. 

D 

Min  8-9 

Month 

8-9 

The  comparable  set  of  distributions  using  only  the  phase  data  in¬ 
volves  a  special  Clint  run.  Select  only  those  form  300  records  (sero 
in  Col.  80)  hsvlng  a  P  In  Col.  34  (inspection  data)  or  an  M  in  Col.  52 
(fix  data).  These  are  combined  with  the  sortie  cards  (Col.  80*2). 

It  is  easier  and  simpler  to  select  all  the  sortie  cards  a-d  set  the 
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Analysis  of  Variance  program  to  output  only  non-zero  data  rather  than 
to  Isolate  the  phase  sorties.  This  special  set  of  data  is  then  pro¬ 
cessed  with  Analysis  of  Variance  using  the  same  sortings  and  control. 
The  resulting  distributions  show  only  the  inspection  and  fix  actions 
of  the  Dhased  inspections.  Partial  results  of  such  a  study  are  shown 
in  Table  4, 

Table  4 

PHASED  INSPECTION  ANALYSIS 
(Tine  in  Hours) 


Test 

Period 

Total 

Turnaround 

Phase 

Turnaround 

Difference 

Number 

1 

37.09 

31.40 

5.69 

11 

2 

39.94 

30.07 

9.87 

1 4 

3 

62.81 

24.66 

38.15 

5 

4 

106.50 

47.24 

59.26 

8 

5 

109.90 

37 . 02 

72.88 

9 

6 

80.30 

37.46 

52.93 

50 

7 

60.66 

48.17 

12.49 

31 

Note  the  large  differences  in  test  periods  3-6.  They  suggest  that 
aircraft  are  being  "lost"  somewhere  between  the  flight-line  and  the 
phase  docks. 

It  is  often  desirable  to  know  how  much  of  the  inspection  man-hours 
consuaptlon  is  due  to  the  inspection  and  how  much  to  the  fix.  If  it 
Is  necessary  to  test  statistically  whether  these  have  been  increasing 
or  decreasing  on  a  month- by-month  basis,  there  is  no  alternative  but 
to  run  each  separately  through  Clint.  The  first  run  uses  the  sortie 
cards  (Col.  802)  and  the  P  in  Col.  34  cards.  This  gives  the  inspec¬ 
tion  data.  The  second  run  uses  the  sortie  cards  and  the  M  in  Col.  52. 
Each  Clint  cutput  having  a  P  in  Col.  20  of  the  sortie  summary  cards 
is  sorted  by  month  within  squadron  and  processed  with  Analysis  of 
Parlance  sensing  the  man-hour  field  (Cols.  61-65)  as  the  variable. 

In  this  instance,  since  the  object  is  to  determine  whether  the  distri¬ 
bution  of  means  is  homogeneous,  the  F-test  is  Inspected. 

As  a  general  statement,  the  small  amount  of  data  does  not  en¬ 
courage  much  Investigation  of  the  phase  inspections  in  great  detail. 
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But  occasionally  one  wishes  to  know  which  systems  are  showing  what 
sort  of  fix  actions.  This  is  easily  done  by  processing  the  M  in  Col. 

52  in  the  following  manner.  Using  FREQ,  determine  the  counts  of  the 
5-digit  work  unit  codes  (Cols.  46-50  of  the  80-0  records).  Select 
those  with  sufficient  frequency  to  warrant  further  investigation  with 
the  Compute  Elapsed  Time  program,  then  process  using  Analysis  of  Vari¬ 
ance.  Major  on  the  '.'ork.  unit  code  (Cols.  46  "0),  minor  on  the  how  mal 
code  (Cols.  53-55).  Sense  Cols.  70-72  (elapsed  time)  for  the  variable 
This  output  will  give  the  how  mal  counts  for  each  code,  plus  the  asso¬ 
ciated  fix  times  for  each  work  unit  code.  The  work  unit  code  totals 
are  shown  at  the  bottom.  If  the  data  sample  is  small,  an  80-80  list¬ 
ing  is  frequently  sufficient  for  most  purposes.  In  this  instance  sort 
as  above,  majoring  on  work  unit  code  and  minoring  on  how  mal  codes. 

DEVIATIONS  AND  DEGRADATIONS 

Frankly,  doing  deviation  analysis  on  7.1  training  sorties  is  some¬ 
thing  less  than  exciting.  After  you  discover  for  the  20th  time  that 
ground  aborts  are  resulting  from  faulty  nose  wheel  steering,  dragging 
brakes  and  sheared  starter  shafts,  the  thrill  is  gone.  With  combat 
data  it  Is  an  entirely  different  story,  since  each  deviation  or  degra¬ 
dation  is  critical.  Each  represents  a  missed  or  degraded  mission; 
each  represents  a  failure  of  the  weapon  system  in  a  situation  where 
failure  is  intolerable.  This  is  the  one  time  that  the  game  is  played 
for  keeps.  Hence  each  item  is  . ritleal.  For  that  reason  we  wish  to 
review  each  in  detail. 

The  deviation-degradation  (DEVPFC1  data  have  an  H  in  Col.  80 
(format  U).  The  best  first  approach  is  to  use  the  Compute  Flapaed 
Time  program  and  punch  out  these  data,  sort  by  deviation  code  (Cols. 
7-8),  and  process  using  DEVDEG.  This  output  is  reviewed  for  coding 
errors  which  must  first  be  corrected.  The  correction  le  easy  beca 'se 
the  verbal  explanations,  listed  bv  the  program,  give  the  story  of  what 
happened.  Following  corrections,  the  data  are  sorted:  Major  on  code 
(Cols.  7-8),  intermediate  on  system  (46-50),  and  minor  on  month  (Cols. 
11-12)  and  day  (Cols.  9-10).  The  data  are  again  processed  with  DEVDFG 
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The  mission  degradation  data  appear  at  the  top  of  the  output  be¬ 
cause  tne  degradations  use  a  blank  in  the  deviation  code  field.  The 
deviation  data  follow  in  alphabetical  order. 

The  intermediate  sort  separates  the  data  by  work  unit  code.  This 
makes  it  easy  to  group  them  by  the  appropriate  indenture  level.  The 
verbal  date  give  a  complete  picture  of  what  happened.  When  knowledge¬ 
able  maint  nance  people  inspect  the  verbal  entries,  they  can  qui  y 
isolate  failures  due  to  hardware  deficiency  and  those  most  likely  due 
to  human  errors  (both  pilots  and  mai  tenance  personnel).  The  human 
errors  are  of  paricular  interest  ' ucause  something  can  be  done  imme¬ 
diately  to  alleviate  the  problem. 

Since  DEVDEG  data  are  frequency  co.  ts,  Chi-square  is  the  appro¬ 
priate  statiatical  test.  Either  Analysis  of  Variance  or  FREO  will  pro¬ 
duce  the  input  data  in  appropriate  form.  Combat  data  are  inevitably 
tagged  by  phase  or  test  period. 

If  Analysis  of  Variance  is  used  to  obtain  the  Chi-square  Input, 
the  base-line  data  can  be  included  in  the  punched  out;uc.  This  is 
entered  in  Co’s.  1-4  of  the  Analysis  of  Variance  control  card.  Setting 
switch  D  "off"  produces  the  Chi-aquare  input  deck  (format  X).  The 
baae-line  data  must  be  added  by  keypunching  if  FREQ  has  been  used  to 
obtain  the  Chi-square  input. 

Aa  an  aside,  one  can  get  into  some  interesting,  but  pointless 
discussions  about  the  beat  base-line  to  use.  The  preferred  one  is 
sorties  attempted.  This  ia  generally  computed  bv  subtracting  sorties 

•  ,n»  e!  l»-.l  from  sort  ire  scheduled  ( f  ragged'  •  -u«*  «  .  '•■rMes  t  ♦  rp:  «•,: 

i  r  .i  *-'(.■  i  • . .  '  ..  .  ■ 

■  t  I'm  !  '  v  ■  r >*  :h  "  ’  v  "  .  i  ’•  i  ••  '  .1  :  • 

■  e  .  i  e-!  count  s . 

The  Chi-square  deck  output  by  Analysis  of  Variance  will  he  in 
minor  within  naior  sequence,  l.e.,  deviation  code  vliMn  time  perl  d. 

To  he  sseful ,  the  data  rust  be  resorted  bv  reversing  :aior  and  minor. 

T  .  t- ,  *  t  :  ue  fn.r  (Cols.  1 1—  1 3  > ,  which,  puts  each  tvpe  ,-f  evl- 

alien  *  e*he  .  Within  each  are  the  records  for  the  periods,  order  ed 


In  propet  sequence. 


-32- 


The  Chi-square  input  deck  can  also  be  obtained  by  using  FREQ. 

This  also  provides  the  major  and  minor  entries.  As  indicated,  the 
data  must  have  the  base-line  added  (Cols.  1-6  right  adjusted)  To  do 
this,  sort  on  major  (i.e.,  period)  and,  for  each  major,  there  must  be 
a  gang  punch  in  the  proper  base-line.  The  deck  is  then  sorted  on  the 
minor,  which  orders  data  by  period  within  deviation  code. 

To  process  the  decks  above  with  Chi-square,  make  one  title  card 
(format  CIS)  and  one  control  card  (format  C16) ,  6et  the  appropriate 
switches,  load  and  go.  See  Fig.  7. 

In  analyzing  deviation  data,  particularly  abort  data,  it  is  ad¬ 
visable  to  check  the  magnitude  of  the  deviations  as  well  as  the  trend. 
Each  /"MS  has  a  hardware  base-line  that  is  the  inherent  characteristic. 
Onto  this  is  superimposed  a  human  error  factor.  This  reflects  the 
adequacy  of  the  training,  procedures,  and  supervision.  The  object  is 
to  keep  the  rates  at  the  hardware  base-line.  As  mentioned,  the  verbal 
information  provides  an  excellent  means  of  isolating  the  human  error 
factors  from  the  other  types.  The  maintenance  nondeliveries  (MNDs)  are 
a  function  of  scheduling  -oor  scheduling  (too  clustered,  too  tightly 
spaced)  results  in  high  .TO  rates.  MNDs  tend  to  show  especially  when 
sortie  rates  are  high.  Review  the  data  to  determine  when  the  MNDs 
occurred  and  check  the  associated  schedule  patterns.  To  do  this,  se¬ 
lect  the  sortie  cards  (Col.  80**2,  format  M)  for  the  periods  in  question, 
gang  punch  a  common  tail  number  (i.e.,  tail  number  8888)  into  all  rec¬ 
ords,  and  process  with  the  Flightline  Display  program.  This  will  shew 
the  scheduled  and  flown  pattern  for  each  day  involved.  To  be  more 
elegant,  add  the  Col.  80»H  (deviation  comments,  format  U) .  This  gives 
the  complete  picture  of  the  scheduling  MND  history. 

BATTLE  DAMAGE  ANALYSIS 

The  analyst  will  not  have  much  battle  damage  data  to  work  with 
since  the  probability  of  battle  damage  is,  fortunately,  quite  low. 

When  such  damage  is  incurred,  there  are  three  separate  kinds  of  records 
that  must  be  reviewed: 

1.  Aircraft  losses — the  details  of  this  are  picked  up  from 
operations  debrief. 
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2.  Heavy  damage — the  aircraft  is  turned  over  to  the  RAM  (Rapid 
Area  Maintenance)  team.  Here  the  loss  is  a  matter  of  days 
and  even  weeks.  This  time  needs  to  be  recorded.  The  details 
of  the  damage  can  be  picked  up  from  the  quality  control  rec¬ 
ords. 

3.  Light  damage — the  aircraft  remains  on  the  possessed  aircraft 
list  and  the  squadron  repairs  the  damage.  This  repair  can 
be  identified  and,  hence,  be  used  as  the  beginning  point  of 
the  analysis. 

The  first  step  is  to  isolate  the  sorties  and  recoveries  on  which 
the  damage  occurred.  This  is  done  with  the  instruction  "Select  data 
(format  K)  having  a  zero  in  Col.  80  and  731  in  Cols.  53-55,  sort  by 
time  within  tail  number,  and  list  80-80".  731  is  the  how  malfunctioned 

code  for  battle  damage.  See  Fig.  8. 

The  80-80  list  is  scanned  for  clusters  of  dates  (generally  one  to 
four  days)  that  indicate  the  on-going  repair.  In  scanning  the  list, 
note  the  action  taken  codes  that  appear  in  Col.  51.  Many  of  the  codes 
will  be  P  for  removal.  Note  that  the  associated  Q  actions  are  not 
shown.  Hence  the  analyst  sees  only  half  of  the  maintenance  involved. 
The  reason  is  that  only  the  removal  ia  coded  731,  battle  damage.  The 
proper  how  mal  code  for  Installing  the  good  part  is  799,  no  defect. 

We  select  only  how  mal  code  731. 

The  easiest  way  to  get  the  whole  story  is  to  use  the  data  clusters 
from  the  80-80  listing  to  isolate  the  sortie  involving  the  damage. 
Select  the  master  file  data  beginning  a  day  or  so  earlier  than  the 
start  of  the  cluster  and  ending  a  day  or  so  later.  This  insures  get¬ 
ting  a  complete  turnaround  bounded  by  a  sortie  card  at  each  end.  Make 
a  flight-line  display  of  these  data  and  check  to  see  that  the  complete 
turnaround  is  available.  Circle  the  how  mal  codes  731  in  red  and 
identify  the  associated  sortie. 

The  analysis  can  often  be  made  directly  from  the  display.  Or, 
in  more  elegant  fashion,  the  Col.  2-7  and  9  cards  from  the  Clint  out¬ 
put  can  be  selected  for  a  more  rigid  analysis.  The  following  must 
be  determined: 

Total  aircraft  hours  l~st 
Total  man-hours  consumed 
Average  fix  time,  all  systems 
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2612160126122200  2 
191 10002151105)0  3 
2501 1 13125192000  5 
2501213125132230  5 
2706170127081930  1 
0612003108120200  3 
0812022108120600  3 
0812171108121730  1 
1012122110121615  6 
1012173110121800  1 
1012166110121900  1 
1012190110121920  1 
10121.92110121935  1 
1012200110122020  1 
2909053129091130  1 
10101 >3110101 165  6 
2209220122092300  6 
0201210102132200  2 
0201220302132305  2 
0201230702132600  2 
2109011621090630  l 
2109160121092600  5 
2209000222090010  5 
2209073122090930  6 
2209170122091800  3 
2308166623082165  5 
2608096126081000  1 
2608180626081930  2 
2508091625081015  1 
2508101625081200  1 
2601170126131900  5 
0909190109092000  2 
1 1 1020011 1 1026O0  5 
1210000212100130  6 
1210013612100200  2 
1210162612101555  3 
1210160112102600  5 
1210163112101830  1 
1310000213100805  6 
1310080113100930  1 
1310080113101700  2 
1310080613101600  6 
1310083113101000  1 
1310100113101200  1 
2001073220131630  6 
2308065123080705  6 
2308082623080930  6 
2308101623081130  6 
2608080126081100  1 
2608090126081015  2 
2608096126081300  3 
2608120126081600  1 
2708073127081100  2 
2708120127081600  2 
2908080129081200  2 
0611192106112330  1 
0810180108102600  6 
0810220108102600  1 


1216168 7589FPM700 
1206666 7629FPM095 
12161A67661FPP106 
12141*8 7661FPP106 
1206168 7663FP 
1 206668 7666FPL026 
1206668 7666FPH9 16 
1206668 7666FPB010 
1331066  7666FPF877 
1 3310687666FP80IO 
13310667666FP8010 
1331068 7666FP8010 
1331068  7666FP6010 
1 331068  7666FP8010 
12 16268 7666FPH1 19 
120616B7650FPM1 73 
12061 68 7650FPF 032 
1206 168 7652FPP669 
1206168 7652FPP669 
1 206168 765 2FPP6 69 
12061687673FP8217 
12061687673FPS662 
1206 168 76 7 3FP 
l 206 1 68 76 7 3FP SO 90 
1 206168  767 3FPP505 
1206168 7673FP 
1206' 68  7673FP 
12061 -87673FP 
12061687673FP 
120616B  7673F  P 
12063687673FP8960 
1216168  7680FP 
1216168768CFPS237 
12161687680FPM095 
I2161687680FP0960 
121616876^5  0FPD089 
121 6168 7680FP 8836 
1216168 7680FPR069 
12161687680FPP98I 
1216 168 7680FPHI 19 
1216168 76 80F PM 119 
121616B7680F PP581 
12161*5  7680FPH1 19 
12141,  7680FPH119 

13316*  »680FPR715 
1216166 '680F  P 
1216166  660FP 
1216168 7680FP 
121616S7680FP 
1216168 7680FP 
1216168 7680FP 
1216168 7680FP 
121616B7680FP 
1216166  7680F  P 
1 2 1 6 1 66  7680FP 
1226168 768»PPM237 
1 2262*8  7666FP 
1 226266 T666FPS6 18 


MOM 

|* If  6L  i  JOi  ETI 

16S3SfF{731  001201 2 1 6lt360p6Cl3  221 2JJ036G 
23200 PM731  10166132313190553211 1K60319 
23200 PF731  001251323139002532331KJ0390 
23200 PF731  100501 32 31 3900103233 1MJ0390 
1 123CC0731  100251 3 1302390251 121 16A0239 
23200 PF731  100661323136201532212160362 
23200 PF731  l 00691 1231 36201 73221 2H60362 
111LE PF 731  100031 33 10362003322 12Q60362 
6616FXD731  10076 1 3320366019322 12MA0366 
1 11HPPF731  100051331036600532212660366 
111XLPF731  100031 33 103660033221 2C60366 
U1KEPF731  100031331036600332212060366 
111KFPF731  10002 133 10366002322 12E60366 
111KHPF731  10003 133 10366003322 12C60366 
1 6530GF  731  100601 31 30272060201 11PK0272 
111 66GF731  100 101206128300 22022 2H6Q 283 
11166RF731  1006012061265010201 1 1860 26 5 
111GOSF731  10020131 3036 70 10 3222  IK 60 36 7 
l 1 1 F JGF73 l  10021 131 303670103222 11*0367 
111 OUGF  731  10018! 31 303670093222 1MA036 7 
16310PF731  000521206 12660521 131 1MA0266 
16310QF731  00600133122660801 131 1K60266 
16310QF731  000071331226500120111660265 
23200 PF731  10080132312650202011 1C60265 
1 1 1 MCRF731  10030120612650102011 1G60265 
23000 PF731  1025013231235050 l 1 122MA0235 
l UGUPF731  10003120612360031 121 1FA0236 
1  1 3  39RM7  3 1  10028120612  360161  121  1HA0236 
1 1 1M6RF731  10010120612370101 1211**0237 
16569RF731  10017120612370171121 1N60237 
23200PF731  101001 323139102032332FA0391 
1671CG0731  100201 33602520101 1233EJ0252 
25000 PC731  002001216128606020222L J0286 
23200 PS731  100591 32312850 15 202228 J0285 
1  linear  31  0000812 16128500620222CJ0285 
1631DRF731  00065 133 112850 1520222 JJ0285 
163 10RF7 31  006001 33 1 22 85060 2022 2K JO 28 5 
111 KMPF731  1 002012161 28 5020 2022 2M JO 285 
14310RF731  003221 33 13266081 202226 J0286 
1 l 1 F JGR731  10015131 302860 1 5 2022 2E JO 286 
HIT  JGR731  101801 31 3028609020222HJ0286 
16310RF731  103 16 13311 286079202228 J0286 
1 11F JGR73I  100151 31 3026601520222FJ0286 
1 11FJGR731  1002013*  3028602020222GJ0286 
1UFJFF731  003601 3 1  303650903231 2AJ0385 
75160PF73!  t0010l2l6l2  35002H122CJ0239 
1 1230PF7SI  000631331 12350111 11 22 A JO 235 
1 1230PF731  100501331 123501211 1220J0235 
1 1 1  F  JGF731  000301 3 1  30238030  H2HBJ0238 
1 11 A6PF731  00025121612360121 121 1AJ0238 
1622DQF731  10100133112380331121 1FJ0236 
1 1 1 F JGF731  100601 31 30236060 11 2 HCJ0238 
1HFJGF731  100701  31 30239035  H211AJ0239 
1 11GSGF731  100801 3 130239060 11 21  IB J0239 
16429GR731  100801 3 13026 1060 I1222FJ0261 
16310GF731  100621 31 3030B0622031 10 JO 308 
16700 PF731  002601226228 1060 20222K 60281 
16700PF73I  000201 33 1226 102020222J80281 
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Average  fix  time,  each  system 
Number  of  sorties  battle  damaged 
Systems  most  frequently  affected. 

These  summaries,  of  course,  should  be  contrasted  to  the  totals  and 
averages  of  the  entire  sample  from  which  the  battle  damages  were  drawn. 

In  addition,  it  is  often  interesting  to  isolate  the  associated  oper¬ 
ations  dats  to  determine  locations,  altitudes,  and  conditions  under 
which  the  damage  occurred. 

STATUS  ANALYSIS 

The  concepts  and  intent  of  status  reporting  is  „.:od:  to  provide 
an  index  of  emergency  readiness  of  a  unit.  In  practice,  the  usefulness 
of  the  information  has  fallen  far  short  of  the  intent,  and  only  the 
naive  assume  that  much  meaningful  information  can  be  obtained  from  the 
status  data  analysis.  There  are  three  major  reasons  for  this  inade¬ 
quacy. 

Fltst,  inaccurate  recording  is  tolerated.  Foolish  indeed  is  the 
unit  commander  who  forwards  an  OR  rate  cf  less  than  72  percent;  censure 
is  inevitable,  so  reports  are  falsified.  Second,  our  experience  has 
been  that  it  is  nearly  impossible  for  the  typical  Job  control  to  ac¬ 
count  accurately  for  the  statu'  of  each  aircraft  in  a  squadron. 

Third,  the  current  status  codes  militate  against  getting  meaning¬ 
ful  information.  The  two  largest  contributors  to  non-OR  conditions 
are  unscheduled  maintenance  and  NORS.  Since  the  unscheduled  mainte¬ 
nance  code  has  precedence  over  the  NORS  coding,  no  true  measure  of  each 
is  obtainable.  This  procedure  results  in  a  well-known  method  of  control¬ 
ling  the  reporting,  which  is  seen  in  the  following. 

Assume  that  an  aircraft  is  down  for  three  NORS-G  items.  If  Job 
control  wishes  to  reduce  the  amount  of  NORS-G  time,  it  merely  aaslgns 
some  unscheduled  maintenance  to  the  aircraft.  While  this  1j  being 
accomplished  (preferably  across  12  o'clock  when  the  NORS-G  counts  are 
made),  the  aircraft  is  coded  as  being  in  unscheduled  maintenance.  Thus 
there  is  an  automatic  reduction  of  the  NORS-G  accounting.  The  reverse 
is  also  possible. 
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Because  the  statue  system  is  so  unsatisfactory ,  we  have  repeat¬ 
edly  attempted  to  eliminate  it  from  the  recording  system.  Thus  far 
we  have  met  with  no  success.  The  data  have  become  so  traditional  that 
they  are  impossible  to  eliminate.  Our  solution  is  to  get  them  as 
painlessly  as  possible.  This  is  done  with  the  following  routine. 

Select  the  data  having  a  3  in  Col.  80,  using  Compute  Elapsed 
Time.  Run  with  Analysis  of  Variance. 

Sort,  control  Major  Col.  80  card  code 

Minor  Col.  26  status  code 

Variable  Cols.  70-72  elapsed  time 

Only  the  entries  under  "Totals"  are  considered.  These  show  the  total 
aircraft  hours  lost  to  each  of  the  various  status  codes.  If  desired, 
these  entries  are  divided  by  the  possessed  aircraft  hours  (airframes  x 
days  x  24  hours)  to  obtain  the  percentages  of  lost  hours.  The  OR  rate 
is  calculated  as  follows: 

Total  hour,,  possessed  -  total  hours  lost 
Total  hours  possessed 

The  following  is  offered  for  thoae  interested  in  obtaining  a  more 
accurate  analysis. 

1.  To  obtain  true  unscheduled  maintenance  downtime,  select  the 
red  X  items,  combine  with  the  sortie  records,  and  process 
through  the  analysis  sequence.  Sum  gross  recovery  times. 

2.  To  obtain  the  true  phased  and  hourly  inspection  data,  see 
section  on  phased  inspections. 

3.  To  obtain  true  NORS-G,  use  the  supply  360  data. 

These  three  generally  account  for  better  than  95  percent  of  status 
downtime.  The  additional  elements,  if  desired,  can  be  obtained  by 
analogous  processing  (e.g.,  select  the  797,  798,  801,  and  802  codes  to 
obtain  tech  order  accomplishments).  Since  status  reporting  is  scanty 
for  other  than  the  three  items  mentioned,  use  the  status  analysis  to 
direct  the  effort. 

Despite  this  pessimistic  discussion,  there  are  some  interesting 
ideas  that  have  resulted  from  status  analysis.  Two  of  these  are  not 
well  known  and  hence  are  but  little  understood. 
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First,  each  aircraft  TMS  has  a  basic  OR  rate  determined  by  its 
reliability-maintainability  characteristics.  For  example,  the  F-AC 
has  an  average  of  three  write-ups  per  sortie,  each  job  taking  approxi¬ 
mately  two  hours.  Obviously,  the  F-AC  basic  OR  rate  must  be  much  less 
than  the  A-37A  which  averages  0.50  write-ups  per  sortie,  each  job  re¬ 
quiring  approximately  an  hour.  Hence,  for  the  same  soitie  rates,  the 
two  aircraft  cannot  possibly  have  the  same  basic  OR  rates.  This  intro 
duces  the  second  factor. 

Each  sortie  exposes  the  aircraft  to  a  possible  malfunction.  Con¬ 
sequently,  the  more  sorties  an  aircraft  flies  in  a  given  period,  the 
greater  its  downtime  will  be  for  unscheduled  maintenance  and  scheduled 
inspections.  Hence,  the  OR  rate  will  be  reduced  when  sorties  are  in¬ 
creased.  This  is  because  the  possessed  hours  are  a  constant.  The  re¬ 
sult  is  a  negative  correlation  between  sortie  rates  and  OR  rates. 
Hence,  command  insistence  on  72-percent  OR  for  all  aircraft  under  all 
circumstances  is  not  meaningful. 

If  status  coding  is  to  be  continued,  which  seems  inevitable,  the 
only  meaningful  approach  is  to  establish  norms  based  on  the  TMS  of  the 
weapon  at  various  sortie  rates. 

BASE  SELF-SUFFICIENCY 

The  completely  self-sufficient  base  is  one  that  repairs  100  per¬ 
cent  of  the  reparable  parts  authorized  for  local  repair.  The  objectiv 
of  the  self-sufficiency  program  is  to  make  the  unit  autonomous  in  an 
emergency.  For  a  number  of  reasons — lack  of  parts,  lack  of  equipment, 
lack  of  skills,  to  name  a  few — 100-percent  self-sufficiency  is  fre¬ 
quently  unobtainable.  The  base  self-sufficiency  analysis  is  intended 
to  reveal  the  reasons  why. 

Whenever  a  reparable  part  is  generated,  an  AFTO  form  211  is  com¬ 
pleted.  Copy  1  normally  records  the  flight-line  action.  The  remain¬ 
ing  copies  are  used  to  control  the  repair  sequence.  Copy  2  is  used 
by  Materiel  Control  to  handle  the  bookkeeping  resulting  from  the 
supply-maintenance  interface.  Copies  3  and  A  are  used  to  record  the 
disposition  of  the  reparable.  Copy  3  is  used  to  record  the  inspection 
test-and-checkout  of  the  reparable.  If  the  shop  repairs  the  unit 


-39- 


during  bench-check,  copy  3  is  completed  and  copy  4  is  destroyed.  If 
the  bench  defers  the  repair,  copy  3  is  used  to  record  only  the  check 
and  inspection  action.  When  the  unit  is  repaired,  copy  4  is  completed, 
recording  only  the  repair  action. 

Analysis  of  the  self-sufficiency  problem  is  slightly  complicated 
because  it  is  often  expedient  to  set  up  a  limited  "assembly  line"  pro¬ 
cessing  in  which  a  number  of  units  are  processed  at  one  time.  In  this 
instar ce  only  one  document  is  completed.  The  "Units  Produced"  block 
contains  the  total  number  of  items  repaired.  Because  we  are  generally 
more  Interested  in  the  number  of  units  than  in  the  number  of  man-hours, 
units  produced  is  generally  used  as  the  variable. 

Certain  action  taken  codes  are  restricted  for  the  recording  of 
the  bench  actions.  These  are  as  follows: 

A — Bench  checked  and  repaired 

B — Bench  checked  and  serviceable  (799) 

C — Bench  checked  and  deferred 

D — Bench  checked  and  transferred  to  another  bench  (used  by 

Forward  Operation  Bases) 

1 —  NRTS  (not  reparable  this  station)  the  repair  is  not  authorized 

2 —  NRTS  no  equipment,  tools,  or  facilities 

3 —  NRTS  lack  technical  skills 

4 —  NRTS  lack  of  parts 

5 —  NRTS  excessive  backlog 

6 —  NRTS  lack  of  technical  data 

7 —  NRTS  excess  to  base  requirements 

9 — NRTS  condemned 

The  NRTS  codes  indicate  the  reasons  for  the  non-repair. 

Most  often  the  analyst  wants  to  summarize  the  bench  actions  for 
each  work  center  for  each  month.  These  summaries  can  then  be  compared 
with  the  previous  month’s  data  to  determine  trends.  Although  occasion¬ 
ally  there  is  interest  in  the  number  of  man-hours  consumed  by  bench 
actions,  most  frequently  there  is  concern  with  the  disposal  of  the 
units.  Hence  the  units-produced  entries  are  generally  used  as  the 
variable . 

The  simplest  way  of  obtaining  a  first  approximation  i6  to  instruct 
the  computer  operator  to  "select,  from  the  off-equipment  file  (66-1 
data),  those  records  containing  a  3  or  a  4  in  Col.  80". 
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Sort,  control  Major  Cols.  28-32  work  center  j 

Minor  Col.  31  action  taken 

Variable  Cols.  56-57  units  produced  j 

Process  with  Analysis  of  Variance.  Since  we  have  used  units  produced 
as  a  variable,  the  unit  counts  will  be  shown  in  the  "Totals"  column 
(the  "Means"  column  will  show  the  average  number  of  units  processed 

per  211  document).  See  Fig.  9.  j 

In  monitoring  base  self-sufficiency,  three  sets  of  action  taken  ! 

codes  are  scrutinized  most  closely:  < 

1.  Repaired  (desirable). 

2.  Serviceable  (too  many  of  these  may  indicate  lack  of  adequate  j 

troubleshooting) .  ! 

3.  2-6  (inclusi”e)  not  repaired  because  of  management  inadequa¬ 
cies. 

Comparisons  may  be  made  on  two  ba?e-lines,  either  the  percentage  of 
total  items  processed,  or  the  it*  a  counts  prorated  by  sorties  flown. 

Either  base  may  be  used  as  the  „<ise  line  when  processing  the  data  w<tt 
the  Chi-square  program. 

FINDING  THE  SHORT  RESOURCE 

The  perfectly  balanced  weapon  system  is  not  unlike  the  classical 
one-hoss  shay  in  that  it  has  the  exact  proportion  of  resources  re¬ 
quired  to  meet  its  task.  This  implies  that  the  balanced  system  has 
exactly  the  right  number  of  pilots,  maintenance  technicians,  supplies, 

AGE  and  so  on  for  its  program;  neither  too  few,  which  degrades  mission 
effectiveness,  nor  too  many,  which  is  needless  expense. 

Unfortunately,  because  of  the  shifts  and  oscillations  of  changing 
mission  requirements,  rotation  of  personnel,  and  fluctuating  demands, 
it  is  rare  that  the  typical  unit  achieves  a  reasonable  approximation 
of  balance.  Inevitably,  when  stress  is  placed  on  the  system,  it  is 
discovered  that  one  or  more  of  the  resources  lb  insufficient  to  ro*ct 
the  increase  in  mission  needs.  It  is  particularly  painful  when  tf.ts 
discovery  is  first  made  in  a  combat  setting.  Hence  a  means  is  needed 
of  unearthing  the  short  resource  to  fill  the  hiatus,  when  possible,  or 
when  impossible,  to  insure  that  it  gets  priority  management. 


EAElVUEHC*  COUNTS  N  IOIAIS  VARIANCE 
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To  this  objective  (resource  utilization  determination),  fraction¬ 
ing  the  data  by  the  sortie  is  without  peer,  since  it  is  the  sortie 
that  causes  resource  consumption.  This  is  seen  first  by  the  provoca¬ 
tion  of  demands  for  a  resource,  which  is  followed  by  the  use  of  the 
resource  for  some  period.  So  it  follows  that  if  the  demands  are  in¬ 
creased,  or  if  the  length  of  the  utilizations  becomes  excessive,  re¬ 
sources  will  be  insufficient  to  meet  the  system  requirements. 

Increases  in  resource  utilization  result  principally  from  three 
factors:  increases  in  sorties,  Increases  in  malfunctions,  and  in¬ 
creases  in  job  durations.  Lacking  computer  simulation  capability,  we 
cannot  predict  resource  needs  when  such  increases  occur.  What  we  can 
do,  however,  is  isolate  resources  currently  in  short  supply  and  use 
tnis  list  as  a  basis  for  predicting  what  might  happen.  Note  that  no 
technique  (simulation  or  otherwise)  will  ever  predict  what  will  happen. 
For  example,  suppose  the  question  arises:  What  would  happen  if  I  go 
from  an  average  one-hour  sortie  to  an  average  two-hour  sortie?  It 
would  ba  natural  to  guess  that  no  change  will  take  place  in  the  landing 
gear  write-ups  (its  usage  is  independent  of  sortie  length),  while  en¬ 
gine  write-ups  should  double.  The  guess  would  probably  be  wrong.  As 
a  matter  of  Interest,  one  sample  of  combat  data  in  which  sortie  length 
increased  from  1  to  1 hours  showed  a  decrease  in  all  resource  utili- 
zat ion. 

Admitting,  then,  that  in  many  instances  it  will  be  impossible  to 
predict  what  will  happen,  there  is  still  a  need  to  be  able  to  identify 
the  short  resource  quickly  to  Insure  that  it  gets  special  management 
attention.  For  certain,  we  will  wish  to  look  at  four  resources:  ait- 
crews,  maintenance  personnel,  ACK,  and  supplies. 

Aircrews 

Aircrew  information  may  be  obtained  from  two  sources.  Fitot, 
extrapolate  the  current  aircrew  determination  rules--for  example,  pilots 
will  fly  a  maximum  of  20  combat  sorties  a  month.  This  rule  provides 
for  nonflying  duties,  R4R,  and  so  on.  Divide  the  projected  monthly 
sorties  by  20  snd  compare  with  the  number  of  certified  aircrews.  Note 
that  when  an  aircrew  requires  more  >.han  one  man,  there  is  often  an 


-41- 


imbalance,  i.e.,  an  excess  of  navigators  or  flight,  engineers.  Hence 
the  determination  is  made  on  intact  aircrews,  and  the  short  resource 
is  the  one  that  determines  the  aircrew  count.  Note  that  there  are 
back-up  resources:  the  squadron  commander,  the  director  of  operations, 
and  others,  who  can  on  occasion  be  called  upon  to  till  out  an  aircrew. 

Second,  review  the  aircraft  deviation  data  (select  80»H,  format 
U,  sort  on  deviation  code  Cols.  7-8,  process  with  DKVDEG) •  Check  the 
verbal  information  on  late  takeoffs  and  operations  cancellation  to 
determine  whether  these  were  caused  because  aircrews  were  riot  available. 
If  so,  review  the  daily  sorties  scheduled  and  flowr.  data  (Col.  80s*.!) 
(format  M)  and  the  sortie  comments  (Col.  80=B)  (format  P). 

Malntenan ce  Personnel 

In  predicting  maintenance  personnel  availability,  the  preferred 
first  step  is  to  scan  the  delay  data  in  its  entirety.  I'll  i  s  is  done  by 
selecting  the  Col.  80**  1  (format  L)  records  and  processing  them  through 
Compute  Elapsed  Time.  These  are  Chen  processed  with  Analysis  of  Vari¬ 
ance.  See  Fig.  10. 

Sort,  control  Major  Col.  ?')  squadron 

Minor  Cols,  h  5  —  h t>  delay  code 

Variable  Cols.  70-71!  elapsed  t  l me 

She  N  and  totals  columns  show  the  reative  proportions  of  delays 
for  each  of  the  following  codes: 

MEN  personnel 

AGE  .it'rosp.ue  ground  equipment 

AWT  parts 

To  determine  the  particular  shops  causing  the  delays  tor  personnel, 
select  the  records  containing  MEN ,  Cols.  S  1-V>,  and  list  tit '-SO.  Itie 
delav-oausing  shops  will  appear  in  ids.  S-'-ss. 

This  set  ot  procedures  gives  on  1  -  a  ( oar-e  overview  of  the  pr-.'Ho*  • 
ior  a  more  detailed  picture,  follow  cite  regular  manpower  analysis  se¬ 
quence,  scrutinizing  closely  those  lnst.in.es  where  utilization  rates 
approach  and  or  exceed  It-1  percent.  It  is  mderstoou  that  the  analyst 
will  also  determine  whether  or  not  excessive  -an— hour  , cnssrpt i on  is 


due  to  Injudicious  shift  assignment.  This  is  also  determined  from  the 
manpower  analysis  sequence. 


A  final  source  of  information  is  the  AFM  66-1  bench  repair  actions 
(copies  3  and  6  of  the  211  actions).  Select  the  Col.  80=3  and  Col. 

8Q«4  from  the  66-1  data.  These  should  be  sorted: 

Major  Cols.  28-32  work  center 

Minor  Col,  51  action  taken 

List  80-80 

The  action  taken  code  "3"  NRTS,  lack  of  technical  skills,  indicates  a 
proficiency  level  beyond  the  capability  of  the  particular  shop  in 
question. 

Aerospace  Ground  Equipment 

Getting  a  complete  and  satisfactory  set  of  AGE  utilization  data 
is  both  difficult  and  costly.  Certainly  we  do  not  recommend  continuous 
recording  as  a  method  of  choice.  For  those  wishing  to  try  getting  a 
sample,  the  methodology  is  described  in  Volume  II.  The  following  is 
offered  as  a  less  c  ’fitly,  but  not  so  satisfactory  solution  to  the 
problem. 

The  300  form  provides  for  the  inclusion  of  a  "local  use"  block, 
which  is  generally  used  to  record  the  roan  number  for  a  maintenance 
action.  This  local  use  block  can  also  be  used  to  record  the  cause  of 
the  aircraft  delay;  i.e.,  when  the  maintenance  cannot  be  completed 
because  of  lack  of  AGE,  the  delay  record  is  created  and  the  specialist 
enters  the  AGE  noun  or  mnemonic.  It  is  not  practical  to  establish  a 
uniform  list  of  identities,  nor  is  it  necessary  since  the  number  of 
records  ttOittX  X  ■ 

As  a  first  determination,  select  the  delay  data  (Col.  80*1)  per¬ 
taining  to  AGE  (Cols.  53-55)  and  list  80-80.  Inspect  Cols.  41-45  to 
determine  the  types  of  AGE  involved  and  the  start  and  stop  times  of 
the  AGE  tc  see  if  the  time  of  day  gives  some  clue. 

If  the  above  inspection  warrante  a  more  detailed  exploration, 
additional  information  may  be  derived  by  putting  the  delay  data 
through  the  manpower  utilization  sequence.  Get  the  touchdown  summary 
data  as  usual  (Col.  80-A) .  From  the  AGE  shop,  get  the  daily  AGE 


availabi.ity  and  convert,  to  Col  .  80=E  (manpower  available)  record 
(format  W) .  This  is  most  easily  done  by  keypunching  directly  into  E 
format.  If  so  desired,  it  can  be  keypunched  in  Col.  80=5  (format  E) 
and  computer  edited  to  produce  the  Col.  80=E  data.  The  delay  data 
converted  to  W  formats  (Col,  80“N)  are  produced  by  processing  the 
delay  data  through  Hourly  Frequency  Accumulate  and  Select.  To  do  this 
the  various  entries  in  Cols.  41-45  of  the  Form  300  data  must  first  be 
recoded  to  agree  with  the  nouns  selected  for  entry  in  the  E  card:  . 

For  example,  the  various  entries  indicating  hydraulic  mule  are  all 
recoded  as  "HMULE"  on  both  the  E  and  N  cards. 

In  using  the  Hourly  Frequency  Accumulate  and  Select  program  to 
process  delay  data,  Cols.  1-6  of  the  control  card  are  punched  041045 
(tne  location  of  the  AGE  code),  Cols.  7  12  are  left  blank,  Cols.  79-80 
are  punched  El.  The  data  are  sorted  as  follows: 


Major 

Cols . 

41-45 

ACE  code 

| 

Cols . 

11-12 

month 

1 

Cols . 

9-10 

day 

Minor 

Co  s . 

13-16 

hour-minute 

The  A,  E  and  N  cards  are  then  processed  through  the  Manpower  Utiliza¬ 
tion  program,  and  the  output  scanned  for  clues.  Note  that  here  the 
"utilization"  is  now  a  picture  of  the  nonavailability  of  AGE,  presum¬ 
ably  because  the  demand  has  exceeded  the  availability  level. 

As  with  manpower,  the  bench  repair  data  may  be  scanned  for  the 
action  taken  Code  2  (NRTS — lack  of  equipment,  tools  or  facilities). 

To  do  this,  select  the  66-1  data  having  a  "3"  or  "4"  in  Col.  80. 

Sort  Major  Cols.  28-32  work  cpnter 

Minor  Col.  51  action  taken 

List  8^-Gv 

Supply 

The  short  supply  resource  is  the  most  difficult  of  all  to  isolate 
This  Is  because  the  maintenance  specialist  has  a  number  of  alternate 
mechanisms  for  clearing  a  write-up  other  than  substituting  a  specific 
black  box  drawn  from  supply:  he  can  fix  the  box;  he  can  cannibalize 
from  another  aircraft;  he  can  replace  the  box  with  the  next  higher 


assembly;  o» ,  in  some  instances,  he  can  let  the  write-up  stand.  Thus 
a  supply  demand  is  rarely  at  a  one-to-one  ratio  wit  .  the  write-ups. 
Hence  there  is  r  reason  to  presume  that  doubling  'he  program  will 
double  the  supply  demands.  In  Rapid  Roger,  for  sample,  the  supply 
demand  rates  decreased  as  the  sortie  rates  increa. °d,  rather  than  vine 
versa,  as  might  have  been  expected.  Again,  there  was  strc*nf  ^  ’idence 
that  the  sortie  type  was  having  an  effect.  (One  unkind  individual 
suggested  that  the  decrease  was  caused  because  maintenance  personnel 
were  learning  not  to  depend  on  supply.)  Unfortunately,  until  provision 
can  be  made  to  provide  the  equivalent  of  the  RAND  Base  Stockage  Model 
at  unit  level,  the  isolation  of  the  short  supply  resource  will  be 
difficult . 

Despite  this,  we  need  to  be  able  to  do  something  besides  wring 
our  hands  in  anguish.  Fortunately,  something  is  available — the  supply 
1050  system  that  makes  accessible  all  sorts  of  interesting  summaries, 
accounts  and  reports.  One  of  the  more  interesting  is  the  AFS-52  report 
which  gives,  on  a  daily  basis  if  so  desired,  the  items  causing  NORS 
grounding.  There  is  one  limitation,  however;  the  reported  item  may 
be  the  one  causing  the  NORS-G  condition,  or  it  may  be  the  bits  and 
pieces  that  are  needed  to  repair  the  item.  If  it  is  necessary  to  make 
this  distinction,  check  with  the  work  center  that  does  the  repair. 

One  of  the  unfortunate  shortcomings  of  the  NORS  codes  is  that  they 
apply  almost  exclusively  to  safety-of-f ligh,.  items.  No  cognizance  is 
given  for  a  defective  weapon  release  problem,  but  because  no  safety  of 
flight  is  involved,  it  is  not  identified  as  a  grounded  aircraft.  Hence 
there  is  no  way  of  identifying  the  shortage  of  critical  (i.e.,  mission 
essential)  items.  This  defect  shows  itself  most  ..fullj  in  the  GZA 
lata;  many  sorties  have  to  be  scrubbed  or  else  flown  in  degraded  combat 
status  because  no  proviso  has  been  made  to  include  the  mission  essenti¬ 
ality  factor. 

Providing  the  analyst  has  the  time,  we  suggest  he  first  '•onstruct 
a  list  of  the  type  of  sorties  his  particular  outfit  will  be  required  to 
fly,  and  from  this,  using  the  assistance  of  knowledgeable  maintenance 
people,  construct  a  list  of  parts  that  are  mission  essential  to  each 
sortie  type.  Our  sortie  type  analyses  of  both  Skoshi  Tiger  and  Rapid 


Roger  data  indicate,  unfortunately,  that  the  diffee  that  the  differences  that 
because  of  sortie  type  cover  the  entire  aircraft,  not  just  the  weapons 
package  However,  two  or  three  sortie  types  (i.e.,  regular,  deep 
interdiction  and  escort)  are  generally  all  that  are  needed  to  cover 
most  situations.  It  can  be  expected  'hat  a  good  bit  of  exploration 
and  analysis  will  be  required  before  the  final  list  of  sortie  types  is 
firm.  To  start  this,  do  a  complete  /-  and  8-card  analysis  comparing 
data  by  sortie  type.  Combine  similar  samples  (i.e.,  those  samples 
showing  homogeneity  of  means  and  variances).  The  object  is  to  find 
the  smallest  number  of  unique  sortie  types.  These  are  used  to  estab¬ 
lish  the  mission  essential  items. 

The  mission  essential  evaluation,  unfortunately,  is  not  complete 
and  sufficient  for  all  purp< because  of  the  flying  rules.  For 
example,  the  local  rules  mav  say  that  on  a  certain  type  of  mission  two 
(or  one  to  three)  of  four  operational  inertial  navigation  systems  are 
sufficient  to  fly  the  mission.  In  contrast,  these  same  local  rules 
may  say  that  on  another  mission  type  all  four  inertial  navigation 
systems  must  be  operational.  This  is  not  to  imply  that  such  decisions 
are  capricious;  on  the  contrary,  they  are  almost  inevitably  the  result 
of  some  catastrophe  or  near  catastrophe  which  has  resulted  in  some 
serious  reevaluation.  Whatever  their  origin,  these  ru.lee  do  complicate 
the  problem  of  determining  the  short  resource,  since  not  only  is  the 
question  of  the  type  of  sortie  involved  but  also  the  mission  structure. 

Despite  the  complexity  imposed  by  the  mission  composition,  our 
analyst  has  no  recourse  except  to  compute  for  all  mission  types.  The 
problem  Is  Identical  to  the  redundant  system  problem  on  one  aircraft. 
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ITI.  ANCILLARY  considerations 

NOTES  ON  AN ALY SIS  OF  "AKIANoE  AND  CHI-SQUARE  COMPUTATION 

The  Analyst'  of  Vari^nro  Hippiays  l n  25,  Volume  II)  are  extended 
to  -2  und  5Z  with  a  reason:  All  maintenance  data  are  markedly  skewed. 
This  distribution  of  observations  is  so  skewed  that  means  are  typically 
'located  near  the  70-?5th  percentiles,  rather  than  the  50th  percentile 
of  normal  distribution.  The  effects  of  this  skew  play  havoc  with  the 
variance.  The  problem  is  worsened  because  the  sample  sizes  (N)  are 
always  unec,  nl.  This  combination  makes  stat  i  sr  i  <■  i  ,„is  shudder,  and  with 
reason.  Since  skewed  distributions  end  unequal  sample  sizes  are  a  wav 
of  life,  there  is  a  need  for  some  comment  on  how  to  live  with  these 
situations . 

irk 

First,  as  indi-ated  by  the  Donaldson  study,  the  F-test  is  exceed¬ 
ingly  robust  with  respect  to  both  type  I  and  type  11  errors  for  some 

'frfrk 

kinds  of  distributions.  For  lognormal  and  exponential  distributions 
(•..’hich  are  similar  to  many  distributions  of  maintenance  parameters),  the 

irk-kic 

F-test  is  conservative  with  respect  to  tu  . h  types  of  errors.  This 
conservative  feature  ovists  for  auv  distribution  w  h  positive  skewness 
and  kurtosis.  This  indicates  that  the  F-test  cm  very  well  in  identi¬ 
fying  random  variation  among  the  sample  means;  If  the  F-test  is  not 
significant,  the  distribution  is  homogeneous;  that  is,  the  two  samples 
being  compared  are  assumed  to  have  the  same  characteristics. 

Second,  the  Analysis  of  Variance  program  displays  Ns  moans ,  and 
associated  variances.  The  latter  facilitates  testing  discrete  cases 


it 

To  follow  this  discussion,  some  knowledge  of  statistics  would  be 
useful.  A  recommended  text  is  W.  J.  Dixon  and  F.  J.  Massay,  Jr., 
Introduction  to  Statistical  Analysis,  2d  Ed.,  McGraw-Hil’  Book  Co., 

*  c  ,  New  York,  1957. 

** 

T.  S.  Donaldson,  Power  of  the  F-Test  fur  Nonnormal  Distributions 
and  Unequal  Error  .os,  The  RAND  Corporation ,  RM-5072-PR,  September 

1966. 

irkrk 

Type  I  errors  reject  a  hypothesis  when  it  is  true;  type  II 
accept  a  hypothesis  when  it  is  false. 

*  *  *  o 

A  conservative  test  is  one  that  gives  smaller  errors  than  would 
occur  if  the  distributions  art  normal. 
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within  the  sample.  Thus  one  might  is'-'ate  a  particularly  interesting 
pair  of  means,  and  compute  a  t-test  b  tween  the  two  to  find  the  differ¬ 
ence  between  means.  Third,  one  can  test  the  difference  between  variances 
by  dividing  the  larger  by  the  smiller.  This  gives  a  pair  of  tests,  one 
on  means  (t-test)  and  one  on  variances  (F-test).  This  is  not,  unfor¬ 
tunately,  the  equivalent  of  twice  as  many  tests  because  the  mean  and 
the  variance  are  related.  Fourth,  an  additional  test  is  provided;  the 
means  columns  are  tagged  with  the  traditional  one,  two  and  three  aster¬ 
isks  based  on  the  harmonic  mean  of  the  sample.  This  helps  circumvent 
the  problem  of  unequal  Ns.  The  tags  are  based  on  standard  errors  ex¬ 
pressed  as  0.05,  0.01  and  0.001  levels  of  significance. 

There  are,  then,  a  number  of  tools  available  for  dealing  with  the 
problem  of  errors  in  skewed  distributions.  Th«.r°  is  also  another  test 
that  could  be  used:  the  nonparamet r ic  Kruska 1-Wa 1 lacc  one-way  analysis 
of  variance.  Since  this  is  distribution  free,  the  skewness  is  not  a 
problem  We  are  currently  comparing  this  with  the  conventional  one-way 
analysis  of  variance.  If  it  turns  out  to  be  better,  we  shall  program 
it  for  the  l**0l. 

Among  much  of  the  elapsed  time  data  is  a  human  intervention  factor 
rhar  -  -••j-r  contributor  to  t'.c  inequality  of  the  variances.  This 
is  seen  in  Table  5.  The  last  line  of  the  table  gives  the  ratio  between 


Table  5 

VARIANCES  OF  FOUR  DIFFERENT  MAINTENANCE  MEASURES 


High/Low 


Cross 

Turnaround 

f  Delays  ! 

Turnaround 

Man-Hours 

N 

i 

12,905.99 

i 

9973.71 

258.79 

2434. 30 

454 

847.78 

337.80 

176  94 

2485.78 

563 

3,353. 14 

1554.27 

402.68 

4423.89 

139 

3,834.20 

2567.  15 

233.50 

2646.21 

844 

2,359.68 

1050.06 

339.72 

2778.45 

503 

3,555.31 

230‘  50 

259.48 

2765.30 

1308 

260. 10 

43.90 

107.25 

1361.00 

870 

49.30  | 

224.80 

3.80 

3.20 

the  largest  and  anal  lest  variance.  Two  of  the  measures  (gross  turn¬ 
around  and  delays)  are  unacceptably  high.  The  other  two,  net  turn- 
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around  and  man-hours,  arc  sr,iail  enough  to  lead  us  to  conclude  that  a  j 

reasonably  valid  F-test  could  be  obtained.  (As  2  crude  rule-of-thumb, 
if  the  N  is  greater  than  50,  and  the  variance  range  less  than  13  to  j 

l--preferably  less  than  5  to  l--a  satisfactory  F-test  can  be  obtained.)  j 

There  is  an  interesting  observation:  gross  turnaround  is  comprised  j 

chiefly  of  two  e lements--net  turnaround  and  delays.  The  length  of  the  I 

j 

delay,  however,  is  purely  a  human  intervention  factor.  The  delay  ends  ■ 

when  job  control  decides  to  start  the  next  job.  Note  that  it  is  the 

presence  of  delays  that  causes  irratic  variance  behavior.  In  contrast, 

net  recovery  is  essentially  a  hardware  characteristic.  Its  duration 

ir  principally  a  function  of  the  weapon's  inherent  characteristics. 

Man-hours  is  also  a  hardware  oriented  quant i ty- -net  fix-time  multiplied 
by  team  size.  The  interesting  observation  is  that  the  human  interven¬ 
tion  factor  seems  to  intrude  a  turbulance  into  the  comparatively  stable 
hardware  characteristics. 

A  similar  set  of  comparisons  is  made  on  the  same  data  to  determine 
the  effects  of  removing  the  NORS  and  the  phased  inspection  data,  as 
shown  in  Table  b.  Here,  the  removal  of  the  NORS  data  substantively 


Tabic  6 

NORS  AND  PHASE  EFFECTS 


All  Data 

1 

|  NORS 

i 

1 

Out 

- -  — y 

Phase  Out 

Both  NORS  and 

Phase  Removed 

l; 

,905. 

94 

205. 

,71 

13.239. 

12 

185. 

99 

&A7. 

78 

222, 

,90 

"is, 

U  J 

207. 

70 

3 

,  353. 

14 

180. 

56 

3,188. 

27 

89. 

33 

3 

,834, 

20 

446 , 

,21 

2,705. 

19 

306. 

50 

2 

,359. 

68 

232. 

.57 

1 ,090. 

13 

155. 

90 

3 

,555. 

31 

1523. 

.09 

2,587, 

01 

1447. 

61 

260. 

10 

'  187. 

82 

172. 

76 

131. 

32 

High/ Low 

14. 

80 

J _ 8, 

,  20  i 

18, 

i  13. 

1 — ^ 

10 

improves  the  variances;  but  the  removal  of  the  phase  data  worsens  the 
picture.  This  is  interesting  because  the  phase  turnaround  analyses 
reveal  a  particularly  maladroit  handling  of  the  phase  turnarounds 
which  ranged  from  a  high  of  9  to  a  low  of  1  i,  days.  This  latter  figure 
is  the  only  acceptable  one,  and  it  occurred  in  only  one  of  the  samples. 


To  those  not  familiar  with  NORS  data,  the  chief  element  in  the 
NORS  turnarounds  is  the  "hangar  queen."  All  the  grounding  NORS  Items 
are  consolidated  against  one  or  more  aircraft  (queens) ,  which  may  stay 
grounded  for  days  or  weeks  at  t.  time.  Terminating  the  NORS  turnaround 
is  also  a  human  intervention  factor.  The  Chief  of  Maintenance  becomes 
weary  of  the  hangar  queen  and  orders  it  ended.  It  is  refurbished  by 
cannibalizing  from  other  aircraft. 

There  are  advantages  and  disadvantages  to  both  consolidation  and 
cannibalization.  Consolidation  is  simple,  but  prolonged  nor.usage  of 
the  aircraft  causes  problems  in  its  hydra  lie  and  electronic  systems. 
Cannibalization  puts  many  aircraft  out  of  action,  each  for  only  one  or 
two  items.  Either  way  you  lose.  As  a  general  statement,  it  is  better 
to  consolidate,  despite  the  additional  cost  in  man-hours  if  the  sortie/ 
aircraft  ratio  is  high.  If  manpower  is  short,  the  reverse  is  true. 

CHI-SQUARE 

ine  Chi-square  program  complements  the  Analysis  of  Variance  pro¬ 
gram.  Whereas  analysis  of  variance  statistically  tests  quantitative 
interval  data  (man-hours,  elapsed  time,  and  so  on),  chi-square  analysis 
serves  the  same  function  with  frequency  count  data  (write-ups  per 
sortie,  break-rates,  abort  rates,  and  so  on). 

In  making  t he  chi-square  determination,  the  switch  D  sett'ng  must 
correspond  to  the  type  of  data  being  processed--ei ther  dichotomous  or 
nondichotomous  information.  The  chi-square  computation  differs  with 
each  type.  Dichotomous  data  require  Yes-No  answers;  the  system  broke-- 
Yes?  No?;  the  aircraft  aborted- -Yes?  No?.  In  statistical  terms  such 
data  take  the  form  p*l-q  (p*yes,  q«no) ,  The  proper  computation  of 
dichotomous  data  includes  the  computation  of  the  nonoccur tence  fre¬ 
quencies.  Thus,  in  the  case  of  system  break-rates,  ‘he  final  chi-square 
includes  a  computation  bJsed  on  the  number  of  times  the  system  breaks 
and  also  one  based  on  the  number  of  times  it  does  not  break.  This 
combined  computation  la  automatically  produced  when  switch  D  Is  "off". 

Nondichotomous  data  ar»  not  of  the  p*l-q  form.  For  exasg>le,  "How 
many  round*  hit  the  target  in  this  strafing  pass?",  or  "Fw  many 


write-ups  otu:  red  on  this  sortie?"  Such  data  take  the  form  p^l-q, 

■  nd  nc. (Occurrence  1  “quencies  must  be  excluded  from  the  chi-square 
computations.  The  _  jpi_r  switch  D  setting  is  "on”. 

The  strafing  pass  provides  an  excellent  illustration  of  t tie  dis¬ 
tinction  between  dichotomous  and  nondichotomous  data.  If  the  method 
of  scoring  is  "Yes,  I  did  hit  the  target,”  or  "No,  1  did  not  hit  the 
target,”  the  data  are  in  dichotomous  form  and  the  chi-square  compu¬ 
tation  must  include  the  nonrccur '•ence  counts.  However,  if  the  method 
of  scoring  is,  "How  many  rounds  did  I  get  in  the  target?",  the  data 
are  not  in  dichotomous  form  and,  accordingly,  the  nonuuurrcr.ee  fre¬ 
quencies  are  excluded  from  the  computations. 

Since  most  measures  are  frequency  counts  (the  majority  of  quan¬ 
titative  data  fall  under  the  aegis  of  either  hours  or  distances),  the 
Chi-square  program  will  be  in  .requent  demand.  To  reduce  the  labor 
in  getting  the  data  into  useable  form,  the  Via  lysis  of  Variance  and 
FREQ  programs  have  been  modified  to  output  the  require!  records.  The 
simplest  is  FREQ.  To  use  this,  sense  for  the  desired  majors  and  minors. 
Set  the  appropriate  switches  to  produce  the  desired  output. 

Since  FREQ  does  not  include  the  base-line  data,  these  must  be 
added  to  the  output  records.  If  the  record  is  on  cards,  they  may  be 
keypunched,  the  automatic  duplication  feature  greatly  facilitates 
this  process.  Since  the  record  volume  of  the  Chi-square  progtam  input 
is  quite  small  (a  complete  file  of  break-rate  trend  data  rarely  exceeds 
100  records),  card  input  is  llmost  always  used  because  of  its  greater 
flexibility. 

lu  using  the  Chi-square  program,  the  following  fields  are  always 
sensed : 

Base-line  (sorties  flown,  items  attempted) 

Item  counts  (write-ups,  breaks,  demands,  hits) 

Minor  code  (3-character  data  Identifier) 

Major  code  (aamplc  identifier) 

The  item  counts  are  always  given  as  whole  numbers,  never  as  decimals 
or  multiples  of  10.  Violation  of  this  rule  causes  erroneous  chi-square 
calculations.  In  contrast,  the  base-line  dats,  which  are  used  only 
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to  compute  the  theoretical  frequencies  (by  prorating),  may  be  scaled  in 
any  manner  desired.  Since  sorties  flown  is  by  far  the  most  frequently 
used  base-line,  the  program  assumes  that  the  base-line  data  have  no 
decimal  place. 

The  reader  may  wonder  why  chi-square  does  not  have  as  many  limita¬ 
tions  as  analysis  of  variance  (e.g.,  unequal  samples,  unequal  variances, 
nonnormality  of  distribution).  The  reason  is  that  chi-square  belongs  to 
a  class  of  statistical  techniques  called  r.onparametric  or  distribution- 
free  methods.  These  have  been  developed  specifically  to  circumvent  the 
restrictions  of  the  parametric  methods.  In  general,  the  nonparametric 
methods  are  not  quite  as  efficient  as  the  other.  They  are,  however,  simple 
to  use  and  understand  and  are  free  of  limiting  constructions. 

MULTIPLE  CORRELATION  ANALYSIS 

Those  few  people  who  have  had  a  chance  to  apply  multivariate  tech¬ 
niques  to  maintenance  data  realize  what  powerful  tools  these  methods  can 
be.  ihere  are,  however,  a  number  of  restrictions  on  the  use  of  these 
techniques,  and  the  nonknowledgeable  find  it  comparatively  easy  to  make 
some  exceedingly  absurd  statements.  We  are  attempting  here  to  offer  a 
somewhat  less  elegant  approach  that  can  be  adapted  to  the  small  business 
computer.  Our  objective  is  to  give  the  analyst  a  means  of  winnowing  his 
data,  isolating  those  elements  that  can  profitably  be  explained  in  more 
detail.  The  method  stops  short  cf  computing  a  full-blown  regression  equa¬ 
tion,  but  it  does  point  out  the  most  likely  member  for  such  an  equation. 

Our  objective  is  to  relate  the  dependent  variable,  Y,  to  as  many  indepen¬ 
dent  variables,  Xi's,  Xj's,  as  are  useful.  The  correlation  matrix,  thus, 
is  scrutinized  to  isolate  Independent  variables  that  correlate  low  with 
each  other  and  high  with  the  dependent  variable. 

To  illustrate  the  above,  consider  first  the  definition  of  a  correla¬ 
tion  area  Xi  Xj  in  which  XiXj*0;  tiiat  is,  Xi  and  Xj  are  not  related: 
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Then  consider  the  correlation  area  of  YXj,  where  there  is  a  correla¬ 
tion  of  0.90: 


Had  we  constructed  a  three-dimensiona 1  model  of  the  relationships,  it 
would  have  looked  like  the  combination  of  our  two  two-dimensional 
models--a  discus-shaped  object  with  the  Y  axis  running  through  it  like 
the  spindle  of  a  top.  The  direction  that  the  spindle  points  to  is  well 
defined  because  of  the  circular  shape  of  the  Xi  Xj  plane.  The  plane 
is  determined  by  rotating  the  discus  to  its  smallest  two-dimensional 
view  ( i.e .  ,  directly  edgewise);  the  Y  axis  is  at  right  angles  to  the 
smallest  two-dimensional  view.  Since  the  object  is  discus-shaped,  the 
position  of  the  Y  axis  is  readily  discovered. 

Now  consider  the  case  when  Xi  Xj  are  highly  correlated  (say  0.80): 


N  / 


Then  construct  an  imaginary  three-dimensional  model  as  before.  The 
result  is  the  shape  of  a  slightly  flattened  cigar.  Now  when  we  attempt 
to  define  the  Xi  Xj  plane  by  rotating  it  to  the  smallest  two-dimensional 
view,  we  can  locate  this  plane  only  with  great  difficulty;  hence  we 
cannot  be  sure  just  where  the  Y  axis  points.  And  it  is  possible  that 
the  smallest  amount  of  data  error  could  materially  alter  the  definition 
of  the  Xi  Xj  plane  and  hence  the  location  of  v.  Thus  we  seek  to  find 
independent  variable  Xi's  that,  are  only  slightly  related  to  each  other 
but  highly  related  to  the  dependent  varx-ole. 

This  is  the  essence  of  the  technique;  the  program  first  computes 
the  matrix  of  correlations  and  then  selects  the  highest  correlation 
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between  the  items  selected  as  the  dependent  variable  (Y)  and  Xi.  It 
checks  the  correlation  between  Xi  and  all  other  Xi’s.  The  analyst  is 
r.ow  in  a  position  to  choose  the  ir.deper.d~rt  "arlahles  that  meet  the 
criteria  indicated  above. 

VARIANCES.  SIGMAS  AND  NORMALIZATION 

In  the  short  space  of  this  Memorandum,  it  is  impossible  to  dis¬ 
cuss  adequately  the  behavior  of  stochastic  phenomena  and  how  to  con¬ 
trol  (measure)  them.  Nevertheless,  the  individual  attempting  to 
analyze  weapon  system  behavior  needs  to  have  some  understanding  of 
the  nature  of  the  stochastic  beast. 

The  word  stochastic  (which  will  net  be  found  in  most  dictionaries) 
implies  variability.  The  fix  time  of  the  radar  system  varies.  It 
varies  over  enormous  dimensions  from  as  little  as  five  minutes  (in  the 
case  of  circuit  breaker  overloads)  to  several  days.  And,  although  the 
average  fix  tire  is,  say  two  hours,  the  variability  above  and  below 
this  average  is  tremendous.  Because  of  such  variation,  it  seems  im¬ 
possible  to  determine  whether  or  not  the  radar  fix  times  have  changed. 
Certainly,  only  the  very  brave  or  very  naive  would  attempt  to  make 
definite  statements  about  such  a  vacillating  phenomenon.  Fortunately, 
thanks  to  the  efforts  of  a  number  of  people  (generally  referred  to  as 
theoretical  statisticians),  the  possibility  of  making  sense  of  the 
phenomenon  is  more  optimistic  than  it  appears. 

The  first  optimistic  note  comes  when  it  is  understood  that,  des¬ 
pite  the  wide  spread  of  the  fix  time  distributions,  the  means  of  the 
various  samples  vary  over  a  much  smaller  range.  For  example,  a  series 
of  weekly  or  monthly  average  fix  times  will  seldom  fall  more  than  20 
minutes  above  or  below  the  grand  mean  of  the  data  sample.  (It  is  worth 
an  aside  to  note  that  this  restricted  ranging  of  the  sample  means  is 
of  small  consolation  to  job  control  personnel  who  need  to  know  the 
ETIC  (estimate  time  to  in- commission)  of  a  particular  system  on  a 
particular  aircraft.) 

The  heart  of  the  entire  statistical  determination  concerns  deter¬ 
mining  whether  the  means  of  the  weekly-monthly  samples  have  shown  only 
the  normal  random  variation  characteristic  of  ail  stochastic  phenomena 


or  whether  the  variation  of  the  means  has  exceeded  the  bounds  of  pro¬ 
priety,  as  the  result  of  some  extraneous  force  event.  For  example,  a 
big  influx  of  trainees  might  be  expected  to  cause  sharp  increases  in 
repair  times.  If  it  is  indeed  true  that  trainees  do  take  longer  fix 
times,  then  we  should  see  the  mean  of  this  data  sample  swing  outside 
the  bounds  of  random  variation.  The  way  of  determining  this  "out-of- 
boundedness"  is  one  of  the  functions  of  statistical  testing.  The 
heart  of  all  the  testing  involves  determining  the  variance,  which  is 
a  measure  of  the  amount  of  spread  or  range  of  a  data  sample. 

The  computation  of  a  sample  variance  is  simplicity  personified. 
Imagine  a  sample  of  radar  fixes  with  a  mean  of  2.0  hours.  Next,  take 
the  list  of  raw  scores  (individual  fix  times)  and  assume  the  first  is 

4.5  hours.  Get  the  deviation  by  subtracting  the  first  raw  score  from 

2 

the  mean  (4.5  -  2,0  *  2.5),  then  square  it  (2.5  ■  6.25).  Do  this  with 

each  raw  score.  Add  up  the  list  of  squared  deviations  to  get  the  sum- 

2  2  2 
of  the  d  (written  2d  ).  Obtain  the  average  d  by  dividing  through  by 

the  number  of  raw  scores--more  properly,  divide  through  by  one  less 
than  the  number  of  raw  scores.  Note  that  the  squaring  process  elimi¬ 
nates  negative  signs.  We  now  have  the  sample  variance.  An  example 
illustrates  the  entire  process. 


Sums 


Raw  Score 

Deviation 

L 

a 

4.5 

2.5 

6.25 

0.5 

-1.5 

2.25 

0.5 

-1.5 

2.25 

1.5 

-0.5 

0.25 

3.0 

1.0 

1.00 

10.0 

2d2  -  12.00 

Mean  «  10.0/5 


!.0. 


Variance  *  12.0/4  ■  3.0  (variance  ■  2d  /n-1) 


The  variance  of  3.0  combined  with  the  mean  of  2.0  gives  us  a 
complete  description  of  the  sample  in  that  we  have  its  "most  repre¬ 
sentative  score"  (mean)  and  an  index  of  the  spread  of  the  data  around 
that  mean.  This  statement  is  true  only  for  nonskewed  distributions. 
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Another  common  measure  of  sample  variation  is  the  standard  devi¬ 
ation,  also  called  sigma.  This  is  the  square  root  of  the  variance,  in 

/ - 

this  instance  yl. 0  ■  1.72.  Note  that  t^:-  takes  us  almost  back  where 
we  started.  We  took  the  deviations,  squared  them,  got  the  ’’average" 
squares,  and  then  took  the  square  root  of  this  average.  Hence  our 
standard  deviation  is,  in  a  way,  the  most  typical  deviation. 

In  illustrating  the  concept  of  variance,  we  used  a  sample  and  its 
mean.  It  is  also  possible  to  take  a  sample  of  means  and  treat  it  the 
same  way.  For  example,  we  might  have  the  monthly  means  for  a  six-months’ 
sample  of  radar  fix  times: 


Means 

Deviations 

d2 

2.0 

0 

0.00 

2.3 

0.3 

0.09 

2. 1 

0.  1 

0.01 

1.7 

-0.3 

0.09 

1.5 

-0.5 

0.25 

jLJ* 

0.4 

OJi 

12.0 

2d2 

-  0.60 

Mean  -  12.0/6  -  2.0. 

Variance  »  0.60/5  “  0.12. 

Given  the  mean,  the  variance,  and  the  tests  for  nonrandomness  (generally 
referred  to  as  nonhomogeneity) ,  the  various  specific  statistical  tests 
are  quickly  made.  For  example,  the  famous  F-test  in  analysis  of  vari¬ 
ance  is  computed  by  dividing  the  mean  variance  (shown  in  the  second 
sample)  by  the  sample  variance  (within  variance),  and  is  computed  much 
like  the  first  sample  above. 

It  is  helpful  to  know  one  more  concept:  Normalization  is  the 
process  of  expressing  the  sample  data  in  terms  of  their  standard 
deviation.  This  is  done  simply  by  dividing  each  unsquared  deviation 
by  the  standard  deviation.  For  example,  a  deviation  one  sigma  above 
t he  mean  would  result  in  a  normalized  score  of  1.0;  a  score  of  1.5 
sigma  below  the  mean  would  normalize  to  -1.5.  If  we  do  this  to  a 
number  of  samples,  we  have  now  reduced  them  to  common  terms  and  can 
properly  make  a  number  of  peculiar  sounding  statements  such  as  "John 
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is  taller  than  Mary  is  musical,"  meaning  that  John's  normalized  height 
measure  is  greater  than  Mary's  normalized  musical  ability  measure. 

Most  tests  of  mean  differences  and  correlation  use  this  normalization 
process. 

To  summarize ,  the  capability  to  be  able  to  deal  with  stochastic 
phenomena  is  provided  by  being  able  to  compute  sample  variance.  Since 
almost  all  maintenance,  operation,  and  supply  data  are  stochastic  in 
nature,  the  ability  to  compute  the  variances  is  not  a  trivial  element. 

ON-SITE  ANALYSIS 

In  the  exercises  prior  to  Rapid  Roger,  the  daia  had  always  been 
gathered  and  edited  at  the  site  and,  at  the  end  of  each  exercise  the 
analysis  was  conducted  elsewhere.  It  was  the  experience  of  the  Skoshi 
Tiger  exercise  that  convinced  us  that  this  method  had  serious  short¬ 
comings  in  chat  we  were  discovering,  through  the  analysis  process,  a 
number  of  unknown  factors.  At  this  late  stage  there  was  no  opportunity 
to  determine  through  local  inquiry  the  details  or  to  take  steps  to 
control  any  undesirable  element  that  the  analysis  revealed. 

Although  plans  for  the  Rapid  Roger  exercise  did  not  include  on¬ 
site,  on-going  analysis  (specifically,  the  plans  called  for  collecting 
and  editing  in  the  theater  with  the  analysis  to  be  done  at  Eglin  Air 
Force  Base) ,  during  the  progress  of  the  test  it  became  apparent  that 
such  procedures  were  desirable  and  feasible  using  those  analysis  pro¬ 
grams  that  ad  been  developed  to  date.  Our  brief  experience  in  the 
theater  was  sufficient  to  demonstrate  the  ev«.  rwhe Iming  advantage  of 
on-site  analysis.  The  contrast  was  particularly  marked  during  the 
post-exercise  analysis  at  Eglin  where,  as  usual,  new  factors  were 
unearthed  and,  once  again,  we  were  faced  with  the  awkward  problem  of 
how  to  understand  and  what  to  do  with  the  new  information. 

It  should  be  clearly  understood  that  what  has  been  called  on-site 
evaluation  does  not  imply  analysis  in  vacuum.  The  findings  must  not 
and  should  not  be  kept  bidden  by  the  evaluation  team.  Rather,  any 
interesting,  unusual,  or  puzzling  findings  should  be  taken  to  the 
location  and  discussed  thoroughly  with  the  knowledgeable  personnel: 
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peculiar  NORS  behavior  is  discussed  with  Supply,  Materiel  Control  and 
Job  Control;  turnaround  problems  vi Job  Control;  and  break-rate  data 
with  pilots,  debrief,  and  the  associated  work  centers. 

All  of  these  people  will  have  valuable  additional  pieces  of  infor¬ 
mation  that  materially  contribute  to  understanding  the  phenomenon. 
Occasionally,  and  this  is  the  part  that  needs  to  be  probed  repeatedly 
and  consistently,  they  will  come  up  with  items  of  intelligence  that 
cause  a  complete  revision  of  the  interpretation.  For  example,  at  one 
base  a  discussion  revealed  that  when  someone  is  building  a  mission, 
the  lead  aircraft  is  always  selected  from  the  A  flight,  while  the 
remaining  ones  are  always  selected  from  the  B  and  C  flights.  This 
means  that  one  set  of  aircraft  (the  A  flight)  are  always  flown  at 
constant  throttle,  while  the  others  never  are.  Having  this  informa¬ 
tion  enabled  the  analyst  to  determine  whether  the  nonrandom  selection 
process  had  an  impact.  It  was  the  bull  session  that  provided  the 
information. 

The  point  we  are  making  is  that  the  analyst  may  not  remain  in  an 
ivory  tower.  He  must  spend  considerable  time  pounding  the  flight- line, 
checking  with  the  Tactical  Air  Control  Center,  Job  Control,  Materiel 
Control  Operations,  Supply,  and  the  various  work  centers.  He  must  ab¬ 
sorb  a  tremendous  amount  of  class  knowledge  from  a  wide  variety  of 
disciplines.  This,  in  turn,  means  that,  because  of  the  tremendous 
complexity  of  modern  weapon  systems,  his  education  will  never  end. 

For,  truthfully,  no  one  person  could  ever  hope  to  absorb  it  all.  This, 
of  course,  implies  that  the  Job  will  alwevs  be  a  good  bit  spiel  i  than 
most. 

THE  HEURISTIC  APPROACH 

The  activities  of  most  computing  centers  fall  mainly  into  either 
production  or  exploration.  It  is  this  production  aspect  that  the  lay¬ 
man  thinks  of,  if  he  thinks  about  it  at  all,  when  he  attempts  to  con¬ 
ceptualize  a  computer  operation.  The  buttons  are  pushed,  the  tapes 
.eel,  the  lights  flash,  and  volla--out  cooes  the  answer.  Not  an 
answer,  but  the  answer.  And  so  it  is  with  production  works  that  a 
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standard  program  is  used  with  standard  procedures  to  output  a  stan¬ 
dard  product;  the  monthly  report,  the  payroll,  or  the  account  summary. 
Production  work  is  an  excellent  and  efficient  way  of  using  a  computer, 
and  saves  thousands  of  tedious  man-hours  daily.  It  is  also  very  dull. 

This  is  not  the  way  it  goes  with  analysis,  however,  which  by  its 
nature  must  be  exploratory.  True  the  lights  flash  and  the  tapes  reel, 
but  mostly  what  comes  out  is  not  the  answer  but  an  answer.  In  the 
analysis  game,  both  with  and  without  computers,  what  really  occurs 
is  a  series  of  iterations  during  which  the  analyst  gradually  evokes 
a  clearer  and  clearer  picture  of  the  phenomenon.  To  do  this,  he  must 
take  repeated  slices  through  the  data,  each  one  intended  to  offer  a 
new  perspective.  Cul  de  sacs  (nonprof itab’e  slices)  are  the  rule,  not 
the  exception.  The  sequence  is  generally  like  the  following. 

Reviewing  the  problem,  the  analyst  determines  that  a  particular 
way  of  slicing  the  data  will  give  him  the  truest  picture.  So  he  slices. 
But,  as  inevitably  happens,  this  exposes  to  him  some  elements  not  in¬ 
cluded  in  his  initial  consideration.  So  he  decides,  if  I  now  slice  in 
this  manner,  I  will  understand  the  role  of  these  new  elements.  So  he 
slices  anew.  Eventually  he  comes  to  a  point  of  diminishing  returns: 
the  new  slices  no  longer  give  him  sufficient  additional  information  to 
be  worth  the  effort  or  ex,  >se.  This  means  he  feels  he  understands  the 
phenomenon  sufficiently  to  meet  the  needs  of  the  moment. 

It  is  this  iterative  process,  actually  a  series  of  hypotheses  and 
corrections  of  hypotheses,  that  is  and  must  be  the  analytical  way  of 
life,  at  least  until  we  develop  such  a  complete  understanding  of  the 
interrelationship  of  all  the  elements  of  modern  weapon  systems  that  we 
can  make  a  production  process  of  the  analysis.  This,  certainly,  will 
not  be  in  the  next  few  years. 

Thi6  distinction  between  production  and  exploration  comes  up  in 
another  manner.  Computers  are  costly,  and  production  work,  because  it 
is  predictable,  allows  an  efficient  use  of  the  computer.  Analysis  work 
is  quite  another  goose.  The  analyst  needs  time  to  digest  his  last  input 
before  he  can  make  a  meaningful  second  pass.  And  the  idee  of  the  com¬ 
puter  standing  idle  while  the  anaiyst  thinks  out  his  next  step  is  enough 
to  make  most  budget  people  shudder,  and  rightfully  so.  Consequently, 
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whether  he  likes  it  or  not,  the  analyst  must  resign  himself  to  the  fact 
that  he  is  going  to  have  to  sandwich  his  time  between  that  of  other 
computer  users  because  the  system  cannot  afford  otherwise.  By  the 
same  token  the  shuddering  budget  people  must  recognize  and  provide 
for  computer  analysis  time,  which  by  its  very  nature  will  always  be 
a  periodic  demand. 

This  last  paragraph,  perhaps,  takes  us  too  far  from  our  initial 
purpose-- that  of  disabusing  the  beginning  analyst  of  the  idea  that  the 
process  is  a  fixed,  canned,  one-shot  affair.  It  is  not.  It  never  will 
be.  In  fact,  our  experience  has  been  that  the  chap  satisfied  with  the 
one-shot  pass  generally  does  not  understand  the  problem. 
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Appendix  A 

KEYPUNCH  FORMATS  AND  LIST  OF  FORMS  AVAILABLE  FOR  RAND/TAC  SYSTEM 


For  the  reader's  convenience,  this  appendix  lists  the  forms  in¬ 
cluded  in  Appendixes  B  through  G  of  Volume  I  for  the  RAND/TAC  system. 
Also  included  in  this  appendix  arc  the  Keypunch  Formats. 


Operations  Forms 

CD  form  101 

Sortie  Debriefing 

oD  form  101 

Debriefing  of  Combat 

CD  form  101 

Degradation  Factors  During  Flight  to 
and  from  Target 

CD  form  101 

Degradation  Factors  During  Combat 

CD  form  101 

Combat  Crewmembers'  Comments  and 
Recommendations 

CD  form  101 

BDA 

CD  form  102 

Joint  Services  Anti-Aircraft  Fire 
Incident  and  Damage  Report 

CD  form  103 

Maintenance  Forms 

FAC  Poststrike  Debriefing  Checklist 

MIP  form  305  test/ 

RR  form  300  test 

Maintenance  Data  Collection  Record 

RAND  form  300 

Maintenance  Data  Collection  Record 

RAND  form  302 

Sorties  Flown,  Scheduled  or  Scrambled 

RAND  form  303 

Aircraft  Status  Summary 

RAND  form  305 

Manpower  Availability 

RAND  form  306 

AGE  Utilization 

RAND  form  307 

Mis* ion  Co 

RAND  form  308 

Deviations/ Degradations 

RAND  form  309 

General  Purpose  Information  Record 

RAND  form  30 D 

F-4C 

Supply  Forms 

CD  form  303 

Record  of  Cannibalization 

CD  form  401  (Part 

A) 

Demand  Register 

CD  form  401  (Part 

B) 

Demand  Register 

CD  form  402 

Receipt  or  Cancel  la* ion  Register 

CD  form  403 

MORS  Register 

Pe  r sonr "1  Foi ms 

CD  form  200 

Personnel  Data  Worksheet 

CD  form  201 

Personnel  Information  Data 

CD  form  202 

Supervisor's  Information  Data 

CD  form  204 

Aircrew  Experience  Record 
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Facl 1 .ties  Forms 

Form  F- 1 
Form  F-2 

Form  F-3 
Form  F-4 
Form  F-5 
Form  F-6 
Form  F-7 
Form  F-8 


Keypunch  Formats 


FHit  Program  Input 
rormat  A 
Format  B 
Format  C 
Format  D 
Format  E 
Format  F 
Format  G 
Format  H 
Format  I 


Airfield  F  ilities  Survey--Monthly 
Airfield  Operations,  Safety  and  Weather 
Survey--D;.i  ly 
Motor  Pool  Survey--Weekly 

Electrical  Power  Generation  Survey--Monthly 
POL  Facilities  Survey--Monthly 
Munitions  Facilities  Survey--Monthly 
Supply  Facilities  Survey-Monthly 
Maintenance  Facilities  Survey--Week ly 


Debrief  Summary 

Porm  300  (On-aircraft  maintenance) 

Sortie--Seheduled  and/or  Flown 

Status  Card 

Manpower  Available 

AGE  Utilization 

Mission  Go 

Deviation/Degradation 
General  Purpose  Comment 


Edit  Program  Output 
Format  J 
Format  K 
Format  L 
Format  M 
Format  N 
Format  P 
Format  Q 
Format  R 
Format  S 
Format  T 
Format  U 
Format  V 


Debrief  Summary 
On-Aircraft  Maintenance 
On-Aircraft  Work  Delay 
Sortie  Flown 

Sortie  Scheduled,  Not  Flown 

Sortie  Comment 

Aircraft  Status 

Manpower  Available 

AGE  Utilization 

Mission  Go 

Deviat ion/Degradat ion  Comment 
General  Purpose  Comment 


Program  Output  Summary  Cards 

Format  W  24-hour  Spread 

Format  X  Analysis  of  Variance 

Format  Y  RECSUM 

Format  Z  Frequency  Count 

Format  AA  Of f-Equipnvnt  (AFTO  211)  Manhour 


Program  Control  Cards 

Format  Cl  Lag/Delay 

Format  C2  Analysis  of  Variance 

Format  Cd  Aircraft  histogram 

Format  C4  Tabic  Loading  Stopper 

Format  C5  Manpower  Utilization  Title  Card 


-65- 


Keypunch  Instructions  and  Formats  (Continued) 
Program  Control  Cards  (Continued) 


Format 

C6 

Edit  Program  Gungpuncli  Master  Card 

Format 

C7 

Display  Program  Date  Select 

Format 

C8 

Display  Program  Tape  Output 

Format 

C9 

Tape  Input 

Fo  rmat 

CIO 

General  Title  Card 

Format 

Cll 

Histogram  Title  Card 

Format 

C12 

Field  Selector 

Forma  t 

C  1 3 

e,.  ...... ec  Liapseu  1  lii.e  t  log r am- -Code 

Se  lector 

Fo  rmat 

C  14 

Frequency  Count  (FREO)  Field  Locator  Catd 

format 

C 15 

Chi-Square  Title  Card 

format 

C  16 

Ch l - Squa re  F i >  Id 

Fo  rmat 

C  17 

Correlation  Field  Designation 

Format 

C 18 

Correlation  General  Title  Card 

Fo  rma  t 

C  19 

Correlation  Fie-ld  Designation  Tabie 
Stopper  Card 

Format 

C20 

Correlation  Independent  Variable  Test 

Cent ro 1  Ca r  1 

Fo rmat 

C21 

Analysis  of  Va.iatKo  Header  Card 

Master  Fables 

Format 

Mi 

Worl-  Center  Master 

Format 

M2 

Tail  Number  Master 

Fermat 

M3 

2-Digit  System  Table 

Format 

M-* 

Edit  Program  AGE  Tab  It 

Clint  Program 

Sunm 

ary  Records 

Format 

R 1 

Ai re  rat t 

Format 

R2 

2-Pig:  t  Sy st  re¬ 

Format 

RJ 

work  Cent  -I 

Comb.it  Dragon 

C  a  r  J 

Form  101  Operations  cards 

Format 

CD1 

Sortie  Debriefing 

Format 

CD2 

Debriefing  of  Combat 

Format 

CD  1 

En  Route  Degradation  Factors 

Format 

CD-. 

Degradation  Factors  on  Target 

Format 

CPS 

Flight  Crew  Comments  and 

Rc  comm.end.it  io-is 

Fo  rmat 

CDt> 

RDA 

Format 

CD? 

Hattie  Damage  Assessment  Re-. irks 

Combat  Dragon 

Sup.p 

1  v  Edit  Output  C a r ds 

Fo ! Ta t 

i  D? 

Demands,  Receipts,  ..n<l  can*  e  I  1  .it  ions 

Fo  rmat 

CD9 

Cannibal  iz.it  ion  Tape  Format 

Fo  '  ma  t 

CD  10 

NORS 

Combat  Dragon 

Supp 

iy  E-.li  t  Input  t  arus 

Fo  rmat 

CD  1 1 

Deman. is,  Receipts,  and  C.un.-i  lat  ions 

Forma t 

C  D  i  2 

i  .mrii’.ilu.it  nn  Car!  Format 

Format 

l  D  1  * 

NORS 

A*VWV  fSS  331*00 


STATUS  CARD 


EDIT  PROGRAM  INPUT 


EiilT  PROGRAM  OUTPUT 


EDIT  PROGRAM  OUTPUT 


EDIT  PROGRAM  OUTPUT 


PROGRAM  OUTPUT  SUMMARY  CARDS 


PROGRAM  OUTPUT  SUMMARY  CARDS 


PROGRAM  CONTROL  CARDS 


PROCRAM  CONTROL  CARDS 
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Otin>v  «OG«AM  OATt  SfilCT  COPTIC*  CAIO 


PROGRAM  CONTROL  CARDS 


COMMUTE  ELAPSED  TIME  PROGRAM  -  CODE  SELECTOR 


PROGRAM  CONTROL  CAROS 


PROGRAM  CONTROL  CARDS 


WO»K  CfNTE*  MASTER 


CLINT  PROGRAM  SUMMARY  RECORDS 


COMBAT  DRACON  CARO  FORM  101  OPERATIONS  CAROS 


COMBAT  DRAGON  CARD  FORM  101  OPERATIONS  CARDS 


OtMA^OS,  tfCfim,  AND  CALCULATIONS-  CD  f 
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